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Introduction 


During  the  past  decade,  we  witnessed  an  extraordinary  evolution  in  surgical  care  based 
upon  rapid  advances  in  technology  and  creative  approaches  to  medicine.  The  increased 
speed  and  power  of  computer  applications,  the  rise  of  visualization  technologies  related 
to  imaging  and  image  guidance,  improvement  in  simulation-based  technologies  (tissue 
properties,  tool-tissue  interaction,  graphics,  haptics,  etc)  has  caused  an  explosion  in 
surgical  advances.  That  said,  we  remain  far  behind  scientists  in  applying  infonnation 
systems  to  patient  care.  This  research  effort  has  proceeded  under  the  mantle  of 
“Operating  Room  of  the  Future”  research.  We  replaced  that  theme  with  the  more 
appropriate  “Innovations  in  the  Surgical  Environment.” 

The  content  of  this  Final  Report  contains  infonnation  pertinent  to  continued  activities  in 
relation  to  the  W81XWH-06-2-0057,  “Advanced  Technologies  in  Safe  and  Efficient 
Operating  Rooms”  project.  This  contract  consists  of  a  scope  of  work  that  fits  seamlessly 
onto  a  prior  research  activity  in  the  contract  DAMD- 17-03-2-0001,  “Advanced 
technologies  in  safe  and  efficient  operating  rooms”  work.  The  current  research  project 
activities  are  based  upon  three  pillars  of  research,  OR  Infonnatics,  Simulation  for 
Training  and  Smart  Image.  A  fourth  research  area  was  included  in  the  Informatics  pillar 
during  this  period  of  perfonnance  that  targeted  physical  and  cognitive  ergonomics/human 
factors. 

Two  of  the  Infonnatics  projects  were  closed  last  year.  The  Intra  Perioperative 
Communication  (IPC)  project  has  been  completed;  the  CAST  project  was  replaced  by  the 
Video  Summarization  project.  In  the  Simulation  pillar,  the  Maryland  Virtual  Patient 
(MVP)  project  has  been  concluded  other  than  for  preparation  of  manuscripts  and 
presentations.  Other  sources  of  funding  for  this  project  are  being  sought.  Work 
continues  on  the  other  projects  under  the  terms  of  a  no-cost  extension.  Milestones  and 
tennination  dates  for  these  projects  were  projected  and  reported. 

The  past  year  of  work  focused  upon  conclusion  of  project  work  in  Informatics, 
Ergonomics  and  Simulation.  Additional  publications  were  prepared  and  are  presented  in 
the  Appendix  to  this  report.  The  no-cost  extension  provided  the  research  team  the 
opportunity  to  develop  a  plan  to  carry  forward  “Innovations  in  the  Surgical  Environment” 
for  the  bettennent  of  both  patient  and  healthcare  provider.  The  plan  is  reported  herein. 

Expansion  of  the  Vision:  Innovations  in  the  Surgical  Environment 

On  November  19,  2010,  a  distinguished  group  of  surgeons,  scientists,  academicians  and 
industry  partners  assembled  in  Baltimore,  Maryland  to  consider  the  future  actions 
associated  with  the  Innovations  in  the  Surgical  Environment  (ISE)  conference.  This 
conference  had  been  convened  annually  for  the  previous  six  years  in  an  effort  to  apply 
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advanced  technology  research  to  the  perioperative  environment.  Heretofore,  the  program 
was  supported  by  four  targeted  pillars  of  research:  Infonnatics,  Smart  Instrumentation, 
Simulation  and  Ergonomics/Human  Factors.  The  response  of  participants  to  this 
conference  led  the  sponsors  to  believe  that  the  meeting  has  extraordinary  potential  to 
meet  unmet  needs  of  the  surgical  and  research  communities.  The  agenda  for  this  meeting 
and  list  of  participants  are  presented  in  Appendix  C. 

Foundational  Statement: 

The  ISE  initiative  was  defined  as  a  gathering  comprised  of  experts  from  disparate  fields 
to  address  the  challenges  facing  surgeons  and  other  healthcare  professionals  in  the 
perioperative  environment.  The  ultimate  goal  of  ISE  is  to  improve  safety  and  outcomes  of 
patients  and  caregivers  within  this  environment.  Collaborative,  multidisciplinary  efforts 
through  the  lenses/pillars  of  Surgical  Visualization  (smart  image),  Smart  Instrument, 
Ergonomics/Human  Factors,  Simulation,  and  Bioinformatics  will  be  among  the  focused 
efforts  of  ISE. 

Strategic  Imperatives  -  an  Overview: 

The  foundational  statement  would  provide  the  base  from  which  the  iterative  and/or 
revolutionary  dialog  would  evolve.  A  fonnal  structure  composed  of  any  of  the 
following — officers,  a  board,  committees,  a  small  cadre  of  pennanent  staff,  and  an 
executive  director — to  operationalize  the  function  (i.e.,  define  the  mission)  and  the 
structure  (i.e.,  execute  the  goals,  support  and  extend  events,  serve  as  a  clearinghouse  for 
funding  and  development)  of  ISE  should  be  put  into  place.  Relevant 
stakeholders/constituents  (e.g.,  representative  of  boundary  crossing  -  academic, 
entrepreneurial,  military,  regulatory,  applied  and  theoretical  scientists  and/or  technicians, 
organizations  as  diverse  as  IEEE  and  AORN)  must  be  invested/involved.  An  overall 
framework  should  be  identified  to  provide  ISE  focus  in  terms  of  addressing  innovative 
issues  and  solutions.  A  very  significant  effort  needs  to  be  expended  in  regard  to  correctly 
branding  ISE  so  that  its  transfonnative  vision  is  captured.  The  “problem  space”  must  be 
identified,  then  the  issue  that  occupies  it  defined,  followed  by  characterization  of  the 
components  of  the  topic.  The  creation  of  an  innovative  clinical  culture,  including  the 
setting  of  standards,  must  be  fostered. 


It  was  proposed  that  the  ISE  initiative  be  carried  forward  through  a  series  of  operational 
activities  that  involved  continuous  planning,  engagement  with  industry,  academia  and 
government,  fonnal  organization  and  efforts  to  obtain  funding.  Strategic  thinking  was 
applied  to  the  preparation  of  annual  conference  with  focused  workshops  with  targeted 
goals  subject  to  the  scrutiny  of  expert  panels.  The  innovative  process  would  address  the 
questions  of  whether  ISE  can  create  a  structured,  adaptable  implementation  process  for 
innovation,  whether  ISE  can  shepherd  a  project  from  conception  to  clinical 
implementation  (which  would  require  ISE  possess  capabilities  to  mitigate  or  remove 
obstacles  such  as  funding,  clinician  acceptance,  regulatory  requirements,  etc.),  and 
whether  ISE  should  — rather  than  addressing  the  “full  research  process” — content  itself 
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with  harvesting  technology  components,  in  particular  basic  uncomplicated  components 
that  in  the  main  are  related  to  emerging  technologies? 

Among  other  considerations,  the  group  considered  factors  of  evaluative  standards  and 
relevancy.  Evaluative  standards  would  be  set  by  the  ISE  in  regard  to  assessment  of 
innovation  (such  as  measuring  changes  in  process  rather  than  changes  in  outcome — a 
strategy  that  might  transfonn  the  common  denial  of  bad  outcomes).  Such  standards  could 
also  refer  to  those  by  which  ISE  rates  and  understands  its  own  performance.  Relevancy 
would  be  established  by  seeking  to  become  the  overseer  of  innovation  and  improvement 
for  a  politically  established  organization  such  as  IOM.  Alternative  actions  to  maintain 
relevancy  included  becoming  the  focal  point  of  a  network  of  professional  associations 
and  thus  facilitate  cross-society  pollination  and  collaboration  to  an  ultimate  end  as 
beneficiary  of  the  research  efforts,  or  ISE  alignment  with  a  hot  topic  such  as  health  IT, 
the  current  national  focus,  and  attempt  to  leverage  surgical  innovation  with  smart  health 
prevention  and  prediction. 


The  distinguished  participants  were  distributed  into  expert  panels  for  consideration  of 
ideas  and  for  development  of  recommendations.  A  summary  of  workgroup  deliberations 
is  presented  in  Appendix  D. 

Body 

A.  OR  Informatics 

The  technological  development  of  medical  infonnatics  has  had  profound  impact  upon 
improvements  in  the  delivery  of  patient  care.  Two  publications  prepared  by  this  research 
team  exemplify  how  that  impact  is  achieved.  These  publications  are  presented  in 
Appendices  A  and  B. 

Informatics  subgroup  1.  Workflow  and  Operations  Research  for  Quality  (WORQ) 

The  Perioperative  Scheduling  Study  studied  how  using  post-operative  destination 
information  during  the  process  of  surgery  scheduling  can  influence  congestion  in 
postoperative  units  such  as  ICUs  and  IMCs,  which  lead  to  overnight  boarders  in  the 
PACU.  The  research  team  was  composed  of  Jeffrey  W.  Herrmann,  Ph.D.,  and  Greg 
Brown,  a  graduate  student,  both  with  the  University  of  Maryland,  College  Park.  The  team 
worked  closely  with  Michael  Harrington,  Ramon  Konewko,  R.N.,  and  Paul  Nagy,  Ph.D., 
for  guidance  and  assistance.  This  research  was  summarized  in  a  Ph.D.  Preliminary  Oral 
Exam,  entitled  The  Surgery  Scheduling  Problem,  Block  Release  Policies,  and  Operations 
Research  Applied  to  Health  Care;  by  William  Herring  under  the  mentorship  of  Dr. 
Hermann. 

We  have  developed  a  mathematical  evaluation  model  for  evaluating  congestion  in  post¬ 
operative  units,  including  ICUs,  IMCs,  and  floor  units.  This  model  requires  data  about 
post-operative  destinations  and  length-of-stay  distributions  for  different  types  of 
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surgeries.  We  have  analyzed  data  about  cardiac  surgeries  from  two  years  and  have 
analyzed  UMMC  financial  records  for  all  of  the  surgical  cases  for  fiscal  year  2007.  We 
developed  an  algorithm  for  predicting  bed  requirements  based  on  the  surgical  schedule 
and  have  conducted  a  preliminary  study  comparing  these  predictions  to  other  prediction 
methods  for  two  units.  The  preliminary  results  show  that  the  new  bed  requirements 
prediction  method  is  more  accurate.  We  plan  to  complete  the  study  and  document  the 
results  in  a  technical  report  this  fall.  We  continue  to  refine  and  implement  mathematical 
models  for  evaluating  how  different  block  release  policies  affect  OR  utilization  and  staff 
overtime. 

A  summary  of  doctoral  level  work  perfonned  by  William  Herring  in  support  of  this 
project  is  included  here.  During  the  Spring  2009  semester,  he  conducted  a  thorough 
review  of  the  operations  research  literature  on  operating  room  (OR)  scheduling.  In  the 
course  of  this  review  he  came  across  what  is  believed  to  be  a  critical  and  understudied 
interaction  that  has  been  the  focus  of  research  since  then.  For  many  hospitals,  including 
UMMC,  the  initial  stage  of  the  surgery  scheduling  process  is  the  allocation  of  available 
operating  room  blocks  to  different  surgical  service  lines.  However,  as  the  schedule  for  a 
given  day  evolves,  focus  shifts  to  individual  patients  and  a  new  set  of  challenges  present 
themselves  to  operating  room  managers. 

A  great  deal  of  research  has  been  conducted  on  algorithms  for  scheduling  individual 
patients  into  available  operating  room  space,  and  in  recent  years  a  good  deal  of  attention 
has  been  paid  to  detennining  the  best  ways  to  allocate  operating  room  blocks.  However, 
very  little  work  has  been  done  on  the  interaction  between  these  two  pieces  of  the 
scheduling  puzzle.  In  order  to  systematically  explore  the  policies  that  affect  this 
interaction,  Herring  worked  closely  with  members  of  UMMC’ s  perioperative  staff  to 
observe  all  stages  of  the  scheduling  process  and  develop  a  model  for  how  the  process 
evolves  as  the  day  of  surgery  approaches.  In  developing  this  model,  he  detennined  that 
the  key  policies  that  control  this  interaction  are  the  block  release  policy  (when  OR 
managers  take  unused  space  back  from  individual  service  lines)  and  the  request  queue 
placement  policies  (how  this  space  is  used). 

Herring  developed  a  stochastic  dynamic  programming  (SDP)  formulation  of  the  single 
day  surgery  scheduling  problem  which  incorporates  the  block  schedule  and  allows  for 
flexibility  in  setting  and  testing  the  effectiveness  of  different  block  release  and  request 
queue  policies.  A  key  component  of  the  formulation  is  the  arrival  process  for  the  demand 
for  OR  space  (both  the  quantity  and  the  timing  of  the  demand).  In  order  to  estimate  this 
demand,  Herring  worked  to  get  access  to  data  from  UMMC’s  CDR  and  went  through  a 
training  course  on  pulling  data  tables  from  the  Clinical  Data  repository  (CDR).  Also, 
because  the  optimal  policy  suggested  by  the  SDP  formulation  might  not  be  practical  from 
a  OR  manager’s  perspective,  he  developed  a  simpler  decision-making  model  which  more 
closely  reflects  how  request  queue  decisions  are  currently  made. 

In  September,  he  wrote  a  computer  program  that  solves  small  instances  of  the  SDP  and 
began  exploring  the  types  of  policies  that  the  model  suggests.  In  order  to  compare  the 
optimal  policies  suggested  by  the  SDP  with  more  practical  policies,  the  program  is 
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flexible  enough  to  accept  policy  constraints  and  only  produce  solutions  that  operate 
within  those  constraints.  Since  he  did  not  yet  have  accurate  estimates  of  the  demand  for 
surgery,  these  initial  runs  were  done  with  simple  demand  distributions  and  tested  the 
formulation’s  sensitivity  to  different  types  of  distributions. 

Herring’s  completed  dissertation  reported  a  model  for  the  surgery  scheduling  process  for 
a  large  operating  room  (OR)  suite,  a  problem  which  involves  decision-making  in  a  highly 
stochastic  environment.  A  typical  modeling  approach  for  this  type  of  problem  is  to  use 
what  is  known  as  a  Markov  decision  process  (MDP)  to  analyze  scheduling  decisions. 

Informatics  subgroup  2.  Operating  Room  Glitch  Analysis  (OGA) 

The  OGA  project,  focusing  on  institutional  learning,  examined  the  workflow  around 
perfonnance  indicators  in  the  perioperative  environment  and  building  a  graphical 
dashboard  to  allow  data  mining  and  trend  analysis  of  operating  indicators. 

The  dashboard  was  constructed  using  the  Ruby  on  Rails  web  development  platform  with 
a  MySQL  database  dynamically  driving  the  queries.  An  interactive  graphical  dashboard 
provided  synthesis  around  delays  in  operations  with  multiple  infonnation  visualization 
techniques. 

The  initial  surgical  dashboard  provided  a  strategic  view  of  the  department.  An  extension 
of  the  data  warehousing  layer  is  an  observation  engine  which,  when  user  specified  events 
occur,  will  trigger  processes  ranging  from  communication  (telecom,  email,  etc)  to 
information  exchange  with  other  applications  via  web  services.  The  user  interface 
contains  three  major  components:  1)  A  data  manipulation  layer  which  allows  interaction 
with  the  data  warehouse  and  provides  analysts  with  means  to  create  and  track  new 
metrics;  2)  A  visualization  toolkit  to  create  graphs  and  web  pages  to  display  information 
effectively.  This  includes  the  means  to  create  clickable  and  animated  graphs;  and  3)  A 
simplified  means  to  specify  observers  within  the  business  intelligence  engine  and  canned 
solutions  to  communication  infonnation  when  events  occur. 

Objective  1. 

Complete  a  business  intelligence  engine  to  handle  aggregation  and  manipulation  of  data. 
Future  work  will  entail  the  refinement  of  dashboards  currently  in  beta  testing. 

To  accommodate  the  need  for  data  validation  as  well  as  tactical  infonnation  about  OR 
use  from  day  to  day  an  additional  dashboard  was  developed.  This  dashboard  was 
designed  to  focus  on  case  to  case  problems  of  the  perioperative  environment.  A  greater 
scrutiny  of  daily  performance  and  case  data  will  drive  the  questions  asked  of  the  strategic 
dashboard  which  remains  in  beta  testing.  During  the  year  of  this  report,  the  following 
work  was  completed.  A  tactical  dashboard  was  developed  in  conjunction  with  the 
strategic  dashboard  and  has  been  deployed  to  an  internally  hosted  server.  The  tactical 
dashboard  is  being  used  within  the  perioperative  environment  to  evaluate  OR  utilization, 
scheduling  workflow,  and  case  data  accuracy.  Data  are  being  validated  at  the  case  level 
and  new  processes  for  data  entry  are  being  designed  to  ensure  the  accuracy  of  the  metrics 
within  the  strategic  dashboard.  A  semantic  relationship  query  mechanism  to  facilitate 
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ubiquitous,  dynamic  filtering  of  data  was  developed  for  the  strategic  dashboard.  The 
strategic  dashboard’s  user  interface  has  been  updated  to  include  an  initial  design  for 
filtering  of  infonnation  as  well  as  a  means  to  create  dashboards.  The  perioperative  data 
are  being  validated  through  the  use  of  the  use  of  the  tactical  dashboard.  This  validation  is 
necessary  before  the  release  of  the  strategic  dashboard  as  poor  data  quality  causes 
inaccuracies  in  aggregated  statistics. 


Informatics  subgroup  2.  Ergonomics/Human  Factors 

Additionally,  within  our  established  infonnatics  research,  for  over  two  years  we  have 
continually  identified  the  emergence  of  Ergonomics/Human  Factors  (E/HF)  as  a  major 
sub-component  interest  within  the  existing  aims  of  our  contract.  As  research  progressed 
with  the  overall  Innovations  in  the  Surgical  Environment  program  and  in  particular  with 
the  Informatics  pillar,  attention  was  drawn  to  the  importance  of  these  factors  to  patient 
safety  and  effective  training  in  surgical  procedures.  This  emergence  of  these  areas  of 
interest  and  subsequent  investigation  of  ergonomics/human  factors  has  been  consistently 
and  formally  reported  in  two  of  our  annual  conferences  and  most  recently  in  quarterly 
and  annual  reports  to  USAMRMC.  Several  manuscripts  related  to  our  research  in 
ergonomics  and  human  factors  were  published  during  the  past  year. 

Human  factors  and  Ergonomics  are  two  related  branches  of  study  that  examine  the 
relationship  between  people  and  their  work  environment.  Ergonomics  often  focuses  on 
the  physical  environment  and  the  human  body,  while  human  factors  center  more  on  the 
cognitive  aspects  of  performance — how  an  operator  interacts  with  the  information 
environment.  The  same  ergonomics  and  human  factors  techniques  credited  with  making 
industrial  processes  safer  and  more  efficient  can  be  applied  to  the  analysis  and 
improvement  of  OR  operations. 

Our  Infonnatics  research  pillar  comprises  and  subsumes  the  investigation  of  ergonomics 
and  human  factors.  To  this  point,  our  discussion  of  workflow  has  taken  a  macro  or 
panoramic  view;  for  example,  how  might  we  most  effectively  track  and  bring  together  the 
people  and  assets  necessary  to  ensure  that  a  patient’s  surgical  experience  is  safe  and 
efficient.  Through  our  formal  recognition  of  human  factors  and  ergonomics  within  our 
existing  research  pillars,  we  focus  on  a  more  micro-level  analysis,  such  as  how  the 
physical  interface  between  the  surgeon  and  the  patient  could  be  improved  and  the 
associated  work  space  chaos  and  stressors  of  minimally  invasive  surgery  (MIS)  be 
reduced. 

The  patient  is  the  center  of  the  ORF.  During  MIS,  the  interfaces  between  the  patient  and 
the  surgeon  are  critical  to  both  the  safety  and  quality  of  patient  care  and  surgeon  welfare. 
Patient-surgeon  interfaces  are  complicated  by  compromises  in  equipment  design, 
technology  limitations,  operating  theatre  layout,  and  technical  approaches.  In  particular, 
ergonomic  problems  in  the  MIS  workspace,  such  as  obstructing  catheters  and  cluttering 
tubes,  can  elevate  the  chance  for  contamination,  increase  surgical  risks  to  the  patient,  and 
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reduce  work  efficiency.  Optimal  workflow  during  MIS  stands  to  be  achieved  through 
better  understanding  of  patient-surgeon  interfaces,  both  intracorporeal  and  extracorporeal. 
In  the  ORF,  advanced  technology  could  function  as  a  key  enabler,  allowing  an  optimal 
patient-surgeon  interface. 

Some  of  our  current  work  is  focused  on  establishing  quantitative,  valid  measures  of 
workflow  within  patient-surgeon  interfaces,  identifying  ergonomic  problems  that  result  as 
a  consequence  of  workplace  designs  (e.g.,  arrangement  or  management  of  cables  and 
catheters),  and  demonstrating  key  barriers  to  optimal  workflow  that  present  direct  safety 
and  efficiency  concerns.  One  project  is  based  on  collaboration  between  surgical  experts 
and  human  factors  experts.  Previous  experiences  in  video  capturing  and  analysis  are 
being  used  as  a  basis  for  development  of  workflow  measures  and  identification  of 
ergonomic  inadequacies.  Time -motion  studies  have  been  conducted  to  collect  objective 
data  on  activities  in  the  patient-surgeon  interface.  Conceptual  workplace  layout  designs 
are  being  developed  based  on  objective  data  and  simulations  of  what  workflow  might  be 
if  interfaces  were  optimized. 

Given  the  physical  risks  associated  with  perfonning  laparoscopic  surgery,  ergonomics  to 
date  has  focused  on  the  primary  minimally  invasive  surgeon.  Similar  studies  have  not 
extended  to  other  operating  room  staff.  Simulation  of  the  assistant’s  role  as  camera 
holder  and  retractor  during  a  Nissen  fundoplication  allowed  investigation  of  the 
ergonomic  risks  involved  in  these  tasks.  Specific  tasks  to  be  completed  in  support  of  this 
research  were  identified  as  objectives  of  the  study. 

Objective  1.  Continue  to  develop  an  assessment  of  difficulty  hierarchy  of  Fundamentals 
of  laparoscopic  Surgery  (FLS)  tasks.  This  task  will  require  extended  work. 

Objective  2.  Develop  an  assessment  of  the  effectiveness  of  self-mentored  surgical 
training.  We  realized  the  importance  of  the  fundamental  understanding  about  the 
characteristics  of  the  movement  patterns  utilized  by  expert  laparoscopic  surgeons.  For  the 
early  stage  of  this  particular  research  project,  we  have  started  establishing  quantitative 
and  objective  methodologies  to  identify  these  expert  movement  patterns  which  must  be 
substantially  different  than  the  movement  patterns  used  by  less  experienced  surgeons.  We 
are  also  in  the  process  of  defining  finite  numbers  of  sub-movements  which  may  create 
complex  surgical  movements  by  successive  combination  of  several  sub -movements. 


Informatics  subgroup  3.  Context  Aware  Surgical  Training  (CAST) 

We  proposed  to  design  and  implement  a  prototype  context  aware  surgical  training 
environment  (CAST)  as  part  of  the  University  of  Maryland  Medical  System’s  SimCenter. 
This  system  was  designed  to  explore  the  role  that  an  intelligent  pervasive  computing 
environment  can  play  to  enhance  the  training  of  surgical  students,  residents  and 
specialists.  The  research  built  upon  prior  work  on  context  aware  “smart  spaces”  done  at 
UMBC;  leverage  our  experience  in  working  with  RFID  in  the  DARPA  Trauma  Pod 
program  as  well  as  in  incorporating  Web-based  infrastructure  and  software  applications 
in  academic  and  professional  development  programs.  The  project  was  expected  to  result 
in  a  pilot  system  integrating  one  or  two  training  resources  available  in  the  SimCenter  into 
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a  context  aware  training  environment  that  can  recognize  the  presence  of  a  trainee  and  or 
mentor  and  take  appropriate  action  based  on  known  training  goals  and  parameters.  The 
project  proposed  to  advance  the  knowledge  of  context  aware  training  environments  in  a 
highly  technical  medical  field  and  provide  a  basis  for  incorporating  more  advanced 
technology  assisted  learning  experiences  in  medicine.  This  “smart  environment”  may 
then,  if  successful,  be  scaled  to  meet  the  needs  of  an  operative  environment  where  the 
technological  demands  may  be  the  similar  or  analogous  to  those  seen  in  the  training 
environment.  Ultimately,  the  advanced  training  and  potential  for  use  in  perioperative 
environments  have  a  long-term  end  goal  of  improving  patient  safety  and  adding  to  the 
body  of  knowledge  in  surgical  training.  Initially,  we  saw  a  situation  were  clinicians  in 
training  can  receive  a  tailored  curriculum.  Additionally,  we  envisioned  a  system  that 
offers  real-time  feedback  and  decision  support  and  education  metrics  to  faculty. 

A  key  goal  this  year  was  to  prototype  the  CAST  system,  and  we  defined  a  typical  use 
case  for  our  system.  A  Student  enters  the  simulation  center.  The  system  identifies  the 
student  (for  instance,  using  their  Bluetooth  phone  or  their  badge),  and  does  a  prerequisite 
check  based  on  the  simulator  the  student  wants  to  perform  the  procedure.  Only  if  the 
student  is  done  with  the  prerequisites,  is  he/she  allowed  to  proceed.  When  the  student 
indicates  that  they  are  ready  to  begin,  the  system  starts  capturing  the  external  and  internal 
view  until  the  student  indicates  that  they  have  completed  the  task.  The  captured  video  is 
then  transferred  to  the  video  server  for  review  by  the  instructor.  The  instructor  interface 
allows  the  instructor  to  see  the  entry  logs  of  students  in  tenns  of  when  they  entered  and 
exited  the  center  along  with  the  corresponding  external  view. 

We  employed  the  spiral  prototyping  approach  as  an  experimental  test  bed;  we  designed 
and  implemented  an  initial  system  prototype  that  would  meet  the  above  functional 
requirements.  The  prototype  integrates  two  machines  with  each  simulator  —  a  small 
Nokia  800  device  for  resident  interaction,  and  a  larger  PC  for  video  capture.  Note  that 
this  is  for  the  proof  of  concept.  A  single  small  form  factor  but  computationally  powerful 
machine  could  be  used  instead.  In  fact,  for  virtual  reality  (VR)  simulators  we  expect  that 
manufacturers  could  eventually  integrate  our  system  directly  into  the  computer  that 
drives  the  simulation. 

Our  prototype  used  Bluetooth  for  localization  of  residents  in  the  simulation  center.  It  was 
designed  to  be  modular,  so  that  any  other  technology  (such  as  resident  ID  cards)  could  be 
integrated  easily.  We  also  hosted  training  materials  including  videos  for  FLS,  Kentucky 
and  Rosser  tasks  in  our  system,  and  tracked  student  progress  through  the  chapters 
checked  out.  This  was  used  for  enforcing  prerequisites  when  students  entered  the 
simulation  centre  to  perfonn  procedures.  In  addition  to  enforcing  prerequisites,  there  was 
a  need  for  the  instructors  to  visually  see  what  the  residents  were  doing  during  their 
simulation  procedures.  We  use  N800’s  built  in  camera  to  capture  the  residents’  external 
views.  These  video  feeds  are  then  fed  into  a  central  server  for  review  by  the  instructor. 

For  location  detection,  we  also  experimented  with  using  the  Awarepoint  tags.  Awarepoint 
uses  a  zigbee  based  mesh  network  for  localization  and  exposes  the  location  infonnation 
through  a  web  service.  Our  experiments  indicated  that  Awarepoint  could  provide  us  room 
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level  information,  but  not  anything  finer.  While  this  would  help  identify  if  the  residents 
were  in  the  simulation  center,  it  would  not  help  detennine  which  machine  they  were 
using,  which  was  needed  for  CAST.  We  demonstrated  our  first  system  prototype  at  the 
ORF  workshop  by  going  through  a  typical  student  workflow. 

We  also  focused  on  moving  the  system  from  UMBC  machines  to  the  MASTRI 
infrastructure  where  they  will  be  housed.  We  purchased  a  small  factor  Dell  machine  to  be 
used  for  capturing  internal  views  from  simulators.  Storage  was  purchased  and  added  to 
the  mastri-intemal  server  for  archiving  both  internal  and  external  video  feeds.  Also,  we 
have  integrated  the  student  database  from  the  hospital,  hosted  FLS  and  other  training 
videos  on  the  hospital  infrastructure  and  hacked  internal  views  of  the  simulators.  We 
developed  the  system  to  capture  internal  video  feeds  and  metrics  from  the  following 
simulators;  Promis,  Stryker  and  the  Laproscopic  VR  simulator. 

Current  efforts  have  focused  on  testing  an  initial  deployment  of  the  CAST  system  at  the 
MASTRI  Center.  We  demonstrated  the  system  to  a  set  of  resident  volunteers  for 
feedback  in  a  form  of  Beta-test  of  the  system.  We  set  up  hardware  and  software  to 
include  the  VR  Simulator  as  part  of  the  CAST  system  deployment.  We  got  usability 
feedback  and  fixed  bugs.  A  significant  part  of  the  effort  was  also  spent  in  surveying  the 
state-of-the-art  in  Video/VR  usage  for  surgical  training.  We  identified  a  small  but 
significant  body  of  work  (e.g.  Sinanan  et  al,  Darzi  et  al)  in  checking  the  construct  validity 
of  the  models  for  training  using  these  simulation  tools.  The  typical  approach  is  to  use 
sensors  to  capture  the  kinematics  of  the  tools,  as  well  as  force/torque  measures.  The 
UMBC/MASTRI  team  decided  that  we  would  like  to  focus  on  an  alternate  approach  that 
i)  focused  on  the  video,  not  (initially)  any  other  sensors  and  ii)  tried  to  capture  using 
machine  learning  techniques  the  ability  of  an  expert  surgeon  to  identify  key  events  in  a 
surgery  that  relate  to  outcome  or  skill  assessment.  This  is  a  very  challenging  and  open 
problem.  Key  initial  steps  were  identified  for  initial  implementation  in  the  first  year. 

A  detailed  description  of  the  CAST  project,  “A  Ubiquitous  Context -Aware  Environment 
for  Surgical  Training”,  was  presented  at  the  First  International  Workshop  on  Mobile  and 
Ubiquitous  Context  Aware  Systems  and  Applications  (MUBICA  2007),  August  2007,  by 
P.  Ordonez,  P.  Kodeswaran,  V.  Korolev,  W.  Li,  O.  Walavalkar,  B.  Elgamil,  A.  Joshi,  T. 
Finin,  Y.  Yesha,  I. George. 

NOTE: 

The  effort  to  establish  a  system  for  archiving  imaged  data  from  training  sites  has  been 
attenuated  due  to  advancements  in  archiving  capability  in  off-the-shelf  systems.  The 
focus  of  this  project  now  rightly  shifts  to  video  summarization  by  unique  application  of 
artificial  intelligence  techniques.  Video  summarization  has  extraordinary  potential  for 
streamlining  the  events  in  the  future  perioperative  environment.  Further,  there  are  many 
and  varied  military  applications  from  video  summarization.  The  UMBC  Graduate 
students  currently  working  on  the  CAST  and  the  background  research  effort  for  the  new 
direction  transitioned  out,  as  the  new  direction  is  less  closely  aligned  with  their  research 


12 


interests.  Dr.  Mike  Grasso,  MD/PhD  in  Computer  Science,  joined  the  effort,  and  a  new 
graduate  student  whose  research  focus  will  be  on  the  video  efforts  joined  the  team. 

Informatics  subgroup  3.  Video  Summarization 

Our  overall  goal  is  to  identity  key  portions  of  surgical  procedures  to  aid  in  video-based 
assessment.  To  establish  feasibility,  we  set  out  to  identify  the  critical  view  of  a 
laparoscopic  cholecystectomy.  The  critical  view  is  used  to  identify  the  key  anatomy  after 
major  dissecting  has  been  completed,  but  before  clipping  the  cystic  duct  and  artery. 

During  the  past  year,  we  completed  an  initial  analysis  of  this  problem.  We  compared 
more  than  50  image  features  with  a  distance  metric  to  identify  the  critical  view  of  a 
laparoscopic  cholecystectomy.  We  experimented  with  roughly  50  different  images 
features  and  several  distance  metrics.  Our  initial  results  showed  a  72%  sensitivity  and 
72%  specificity.  The  study  was  small  in  size,  using  only  5  laparoscopic  cases,  and  our 
comparisons  were  limited  to  one  image  feature  at  a  time.  An  abstract  was  submitted  to 
the  American  Medical  Infonnatics  Association  (AMIA)  Fall  Symposium  (appendicized). 

During  the  last  several  months,  we  have  been  working  two  new  initiatives.  The  first  is  the 
creation  of  an  image  classifier  using  a  support  vector  machine  (SVM).  This  is  a  machine 
learning  approach  that  uses  multiple  image  features  to  train  the  image  classifier.  Our 
initial  accuracy  with  the  SVM  improved  to  about  90%.  This  original  SVM  was  built  from 
only  5  laparoscopic  cases.  Our  understanding  is  that  25  additional  cases  will  be  available 
after  IRB  approval  has  been  obtained.  Accuracy  should  improve  when  more  cases  are 
used  to  train  the  SVM.  In  addition,  we  used  particle  analysis  and  edge  detection  to 
identify  key  segments  inside  each  image.  We  also  plan  to  use  this  data  to  increase  the 
accuracy  of  the  SVM.  During  this  past  year,  we  prepared  a  protocol  for  the  use  of  25 
additional  video  cases  upon  which  to  build  the  SVM. 

Informatics  subgroup  4.  Operating  Room  Clutter  (ORC) 

The  Operating  Room  Clutter  project  enters  its  final  phase  under  the  provisions  of  the 
contract,  and  ended  during  the  period  of  perfonnance  reported  here.  Further  research 
activities  will  seek  support  from  other  funding  agencies. 

Prior  to  completion,  the  project  team  worked  on  the  use  of  advanced  video  technology  to 
support  coordination  in  operating  rooms.  Activities  were  in  four  areas.  All  publications 
referred  to  may  be  found  in  the  website:  http : //hfrp . umaryland . edu .  For  full  length  journal 
articles,  PDF  files  may  be  downloaded.  For  others,  abstracts  are  available.  In  all,  we 
published  8  full-length  peer  reviewed  journal  articles,  2  full-length  peer  reviewed 
proceeding  articles,  and  8  conference  abstracts.  The  references  below  can  provide  further 
details. 


A.  Models  of  decision  making  for  operating  room  management. 
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We  reviewed  literature  and  developed  a  synthesis  report  on  the  state  of  the  art  of 
decisions  on  the  day  of  surgery.  Furthennore,  we  developed  models  for  decision 
support  systems  for  operating  room  management.  The  activities  in  this  area  were 
reported  in  the  following  publication: 

1.  Dexter  F,  Xiao  Y,  Dow  AJ,  Strader  MM,  Ho  D,  Wachtel  RE.  Coordination  of 

Appointments  for  Anesthesia  Care  Outside  of  Operating  Rooms  Using  an 
Enterprise  Wide  Scheduling  System.  Anesthesia  and  Analgesia.  105:1701-1710. 
2007  ""  . 

B.  Operating  room  multimedia  system  design  and  methodology. 

We  developed  technology,  primarily  based  on  algorithms  of  video  processing  and 
biosignal  processing,  to  display  status  of  operating  rooms.  The  displays  are  to 
increase  situational  awareness.  The  technological  advances  made  by  our  group 
were  reported  in  the  following  publications: 

2.  Xiao  Y,  Schimpff  S,  Mackenzie  CF,  Merrell  R,  Entin  E,  Voigt  R,  Jarrell  B.  Video 
Technology  to  Advance  Safety  in  the  Operating  Room  and  Perioperative 
Environment.  Surgical  Innovation.  14(1):  52-61.  2007 

3.  Hu  P,  Xiao  Y,  Ho  D,  Mackenzie  CF,  Hu  H,  Voigt  R,  Martz  D.  Advanced 
Visualization  Platfonn  for  Surgical  Operating  Room  Coordination:  Distributed 
Video  Board  System.  Surgical  Innovation.  13(2):129-135.  2006 

4.  Hu  P,  Seagull  FJ,  Mackenzie  CF,  Seebode  S,  Brooks  T,  XiaoY.  Techniques  for 
Ensuring  Privacy  in  Real-Time  and  Retrospective  Use  of  Video.  Telemedicine 
and  e-Health,  12(2):  204,  T1E1.  2006 

C.  Survey  and  descriptive  studies  of  operating  room  management,  with  and 
without  the  support  of  advanced  video  technology. 

In  conjunction  with  technology  development,  we  conducted  observational  and 
survey  studies  of  operating  room  management.  These  studies  and  associated 
results  were  in  the  following  publications: 

5.  Seagull  FJ,  Xiao  Y,  &  Plasters  C.  Infonnation  Accuracy  and  Sampling  Effort:  A 
Field  Study  of  Surgical  Scheduling  Coordination.  IEEE  Transactions  on  Systems, 
Man,  and  Cybernetics,  Part  A:Systems  and  Humans.  24(6),  764-771.  2004 

6.  Dutton  R,  Hu  PF,  Mackenzie  CF,  Seebode  S,  Xiao  Y.  A  Continuous  Video 
Buffering  System  for  Recording  Unscheduled  Medical  Procedures. 
Anesthesiolosv,  103:A1241.  2005 

7.  Gilbert  TB,  Hu  PF,  Martz  DG,  Jacobs  J,  Xiao  Y.  Utilization  of  Status  Monitoring 
Video  for  OR  Management.  Anesthesiolosv .  103:A1263.  2005 
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8.  Dutton  R,  Hu  P,  Seagull  FJ,  Scalea  T,  Xiao  Y,  .  Video  for  Operating  Room 
Coordination:  Will  the  Staff  Accept  It?.  Anesthesiology:  101:  A1389.  2004 

D.  Technology  evaluation. 

We  conducted  evaluation  studies  of  the  technology  deployed.  The  primary  focus 
was  on  user  acceptance  and  usage  patterns.  The  focus  was  chosen  because  the 
current  science  of  operating  room  management  has  concluded  that  improvement 
of  decision  making  on  the  day  of  surgery  will  lead  to  improvement  in  intangible 
outcomes,  such  as  situation  awareness,  and  will  unlikely  lead  to  improvement  in 
operating  room  throughput  (e.g.,  volumes  and  economic  returns).  Our  work  was 
reported  in  the  following  publication. 

9.  Xiao  Y,  Dexter  F,  Hu  FP,  Dutton  R.  Usage  of  Distributed  Displays  of  Operating 
Room  Video  when  Real-Time  Occupancy  Status  was  Available  .  Anesthesia  and 
Analgesia  2008;  106(2):554-560.  2008 

10.  Kim  Y-J,  Xiao  Y,  Hu  P,  Dutton  RP.  Staff  Acceptance  of  Video  Monitoring  for 
Coordination:  A  Video  System  to  Support  Perioperative  Situation  Awareness. 
Journal  of  Clinical  Nursing  (accepted).  2007 

The  project  team  has  worked  on  the  use  of  advanced  video  technology  to  support 
coordination  in  operating  rooms.  We  developed  models  for  decision  support  systems  for 
operating  room  management.  We  developed  technology,  primarily  based  on  algorithms 
of  video  processing  and  biosignal  processing,  to  display  status  of  operating  rooms. 

In  conjunction  with  technology  development,  we  conducted  observational  and  survey 
studies  of  operating  room  management.  We  conducted  evaluation  studies  of  the 
technology  deployed.  The  primary  focus  was  on  user  acceptance  and  usage  patterns.  The 
focus  was  chosen  because  the  current  science  of  operating  room  management  has 
concluded  that  improvement  of  decision  making  on  the  day  of  surgery  will  lead  to 
improvement  in  intangible  outcomes,  such  as  situation  awareness,  and  will  unlikely  lead 
to  improvement  in  operating  room  throughput  (e.g.,  volumes  and  economic  returns). 

Informatics  subgroup  5.  Improving  Perioperative  Communications  (IPC) 

Background: 

In  the  UMMS  OR  the  Cardiac  Surgery  Service  utilizes  a  common  communications  point 
(a  “cardiac  phone  line”)  that  in  a  sense  is  used  to  acquire  information  and  provide  that 
information  to  any  team  member  who  calls  the  line  to  acquire  information.  The  cardiac 
phone  line  has  been  scripted  and  is  actively  in  use  through  a  voice  mail  system.  It  can 
only  be  altered  by  dedicated  personnel  with  password  capability.  The  script  involves  the 
following  standardized  information:  Identification  of  individual  providing  infonnation, 
the  Date  of  surgery,  the  Total  number  of  cases,  and  OR  location,  patient  name,  case 
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order,  medical  record  number,  age,  surgeon,  anesthesiologist  and  procedure.  Evening 
schedule  updates  have  been  made  possible  through  a  second  phone  line  option. 

After  some  effort,  we  can  now  move  to  track  updates  on  the  phone  line  and  correlate 
these  updates  with  OR  start  delays.  Thus,  we  refined  the  IPC  question  to  Does  more 
accurate  infonnation  as  evidenced  by  updates  on  the  phone  line,  ie  improved 
communication,  result  in  fewer  problems  in  the  morning  with  cardiac  surgical  cases 
starting  on  time-  are  instruments  better  prepared  for  the  procedure,  are  operating  rooms 
better  equipped  for  the  appropriate  case,  are  the  correct  pick  lists  utilized  for  the  correct 
surgeon,  is  there  less  of  a  transport  delay  because  the  patient’s  hospital  location  has  been 
identified?  The  question  contains  reference  to  some  of  the  delay  codes  that  are  currently 
utilized  by  the  Operating  room  tracking  system  and  reported  for  glitch  analysis. 

With  the  assistance  of  the  communications  personnel,  we  reconfigured  the  cardiac  phone 
line  so  that  we  can  actually  track  the  phone  calls  made  to  the  phone  line.  This  enabled  us 
to:  Detennine  key  personnel  who  are  utilizing  the  phone  line,  Detennine  groups  of 
personnel  utilizing  the  phone  line  (i.e.  nursing,  anesthesia,  perfusion),  Detennine  which 
groups  are  not  utilizing  phone  line  information  (i.e.  anesthesia  techs),  Determine  whether 
there  is  a  time  variable;  is  there  a  better  time  to  call  for  updates?  Should  updates  be  made 
at  predetermined  times  or  should  they  be  more  dynamic? 

We  hypothesized  that  information  gained  from  increased  communication  improves  OR 
efficiency.  If  this  is  the  case  we  can  then  move  to  see  if  more  real-time  enabling 
technologies  might  be  deployed  to  other  services  within  the  UM  ORs  and  perhaps  other 
ORs  “everywhere”. 

During  work  on  this  project  last  year,  the  team  sought  to  find  an  appropriate  “question” 
with  which  to  focus  this  effort.  In  particular,  there  was  a  need  to  tie  a  perfonnance  metric 
(perioperative  workflow  related)  to  the  IPC  task.  Although  some  progress  was  made,  this 
aspect  of  Innovations  in  the  Surgical  Environment  was  tenninated  and  future  efforts  will 
be  refocused.  Funding  will  be  sought  from  other  agencies 

B.  Simulation 

Simulation.!  The  Maryland  Virtual  Patient 

We  present  here  a  simplified  description  of  the  MVP  simulation,  interaction  and  tutoring 
system.  A  virtual  patient  instance  is  launched  and  starts  its  simulated  life,  with  one  or 
more  diseases  progressing.  When  the  virtual  patient  develops  a  certain  level  of 
symptoms,  it  presents  to  the  attending  physician,  the  system’s  user.  The  user  can  carry 
out,  in  an  order  of  his  or  her  choice,  a  range  of  actions:  interview  the  patient,  order 
diagnostic  tests,  order  treatments,  and  schedule  the  patient  for  follow-up  visits.  The 
patient  can  also  automatically  initiate  follow-up  visits  if  its  symptoms  reach  a  certain 
level  before  a  scheduled  follow-up.  This  patient-physician  interaction  can  continue  as 
long  as  the  patient  “lives.” 
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As  of  the  time  of  writing,  the  implemented  MVP  system  includes  a  realization  of  all  of 
the  above  functionalities,  though  a  number  of  means  of  realization  are  temporary 
placeholders  for  more  sophisticated  solutions,  currently  under  development.  The  most 
obvious  of  the  temporary  solutions  is  the  use  of  menu-based  patient-user  interaction 
instead  of  natural  language  interaction.  While  this  compromise  is  somewhat  unnatural  for 
our  group,  which  has  spent  the  past  20  years  working  on  knowledge-based  NLP,  it  has 
proved  useful  in  permitting  us  to  focus  attention  on  the  non-trivial  core  modeling  and 
simulation  issues  that  fonn  the  backbone  of  the  MVP  system. 

MVP  currently  covers  six  esophageal  diseases  pertinent  to  clinical  medicine:  achalasia, 
gastroesophageal  reflux  disease  (GERD),  laryngopharyngeal  extraesophageal  reflux 
disease  (LERD),  LERD-GERD  (a  combination  of  LERD  and  GERD),  scleroderma 
esophagus  and  Zenker’s  diverticulum. 

At  the  beginning  of  a  simulation  session,  the  system  presents  the  user  with  a  virtual 
patient  about  whose  diagnosis  he  initially  has  no  knowledge.  The  user  then  attempts  to 
manage  the  patient  by  conducting  office  interviews,  ordering  diagnostic  tests  and 
prescribing  treatments. 

Answers  to  user  questions  and  results  of  tests  are  stored  in  the  user’s  copy  of  the  patient 
profile,  represented  as  a  patient  chart.  At  the  beginning  of  the  session,  the  chart  is  empty 
and  the  user’s  cognitive  model  of  the  patient  is  generic  -  it  is  just  a  model  of  the 
generalized  human.  The  process  of  diagnosis  results  in  a  gradual  modification  of  the 
user’s  copy  of  the  patient’s  profile  so  that  in  the  case  of  successful  diagnosis,  it  closely 
resembles  the  actual  physiological  model  of  the  patient,  at  least,  with  respect  to  the 
properties  relevant  to  the  patient’s  complaint.  A  good  analog  to  this  process  of  gradual 
uncovering  of  the  user  profile  is  the  game  of  Battleship,  where  the  players  gradually 
detennine  the  positions  of  their  opponent’s  ships  on  a  grid. 

At  any  point  during  the  management  of  the  patient,  the  user  may  prescribe  treatments.  In 
other  words,  the  system  allows  the  user  not  only  to  issue  queries  but  also  to  intervene  in 
the  simulation,  changing  property  values  within  the  patient.  Any  single  change  can 
induce  other  changes  -  that  is,  the  operation  of  an  agent  can  at  any  time  activate  the 
operation  of  another  agent. 


Simulation:  Utility 

The  MVP  project  can  be  viewed  as  just  one  of  a  number  of  applications  in  the  area  of 
intelligent  clinical  systems.  The  latter,  in  turn,  can  be  viewed  as  one  of  the  possible 
domains  in  which  one  can  apply  modeling  teams  of  intelligent  agents  featuring  a 
combination  of  physical  system  simulation  and  cognitive  processing. 

So,  in  the  most  general  terms,  our  work  can  be  viewed  as  devoted  to  creating  working 
models  of  societies  of  artificial  intelligent  agents  that  share  a  simulated  “world”  of  an 
application  domain  with  humans  in  order  to  jointly  perfonn  cognitive  tasks  that  have 
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until  now  been  performed  exclusively  by  humans.  Sample  applications  of  such  models 
include: 

o  a  team  of  medical  professionals  diagnosing  and  treating  a  patient  (with  humans 
playing  the  role  of  either  a  physician  or  a  patient) 
o  a  team  of  intelligence  or  business  analysts  collecting  information,  reasoning  about 
it  and  generating  analyses  or  recommendations  (with  humans  playing  the  role  of 
team  leader) 

o  a  team  of  engineers  designing  or  operating  a  physical  plant  (with  humans  playing 
the  role  of  team  leader) 

o  a  learning  environment  (where  humans  play  the  role  of  students). 

As  can  be  seen,  this  work  is  at  the  confluence  of  several  lines  of  research  -  cognitive 
modeling,  ontological  engineering,  reasoning  systems,  multi-agent  systems,  simulation 
and  natural  language  processing. 

During  the  period  of  perfonnance,  we  have  been  working  on  the  following  issues: 

1.  We  have  continued  to  develop  a  computational  model  of  the  cognitive  agent.  We 
have  tested  the  goal-  and  plan-based  reasoning  component  and  its  interaction  with  the 
interoceptive  and  language  perception  modules  and  verbal,  mental  and  physical 
action  simulation  modules. 

2.  We  have  spent  much  of  the  time  preparing  for  the  demonstration  of  the  system  at  the 
program  conference  and  at  several  meetings  of  the  American  College  of  Surgeons. 

In  particular,  we  have  developed  a  new  demo  interface. 

3.  We  have  continued  to  work  on  the  natural  language  substrate  of  the  system, 
concentrating  on  enhancements  required  for  processing  dialog  (not  expository  text). 

As  part  of  this  module,  we  have  implemented  an  enhanced  microtheory  of  indirect 
speech  acts. 

4.  We  have  continued  working  reference  resolution  algorithms  (this  is  a  very  difficult 
task  in  and  of  itself). 

5.  We  have  continued  work  on  the  acquisition  of  ontology  and  lexicon  knowledge. 

6.  Improvement  of  the  DEKADE  user  interface  has  continued  apace.  New  facilities  for 
editing  and  viewing  intermediate  and  final  results  of  text  analysis  have  been 
introduced  and  existing  ones  improved. 

7.  We  have  spent  considerable  time  on  improving  the  documentation  of  the  project 
work.  We  have  written  and  submitted  for  publication  5  papers  describing  aspects  of 
our  system. 
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During  this  reporting  period,  the  research  team  continued  to  refine  the  cognitive 
simulation  system  by  adding  more  clinical  scenarios  and  challenges.  The  communication 
between  patient  and  physician  has  reached  a  high  level  of  realism  and  clinical  utility. 

An  extensive  summary  of  this  project’s  work  was  presented  at  the  annual  meeting  of  the 
American  College  of  Surgeons,  at  the  DOD-sponsored  workshop  on  psychometrics  of 
simulation/games,  and  a  similar  DOD  meeting  in  San  Antonio,  Texas. 

We  continue  to  discuss  with  representatives  of  the  DOD  Medical  Departments  the 
application  of  cognitive  simulation  training  for  far-forward  care  providers.  As  this 
project  winds  down,  the  investigators  will  seek  sources  of  funding  to  attain  the  potential 
of  the  cognitive  simulation  system. 


Simulation.2  Training  for  Surgical  Excellence  and  Patient  Safety 

The  development  of  the  Maryland  Advanced  Simulation,  Training,  Research  and 
Innovation  center  (MASTRI)  opened  the  door  to  innovative  research  opportunities  that 
enhance  surgical  training  and  improve  patient  safety.  Within  the  existing  scope  of  the 
current  contract,  several  projects  will  be  undertaken  during  the  final  year  of  the  contract 
that  conceive,  develop  and  validate  simulation-based  training  for  proficiency  in  the 
perfonnance  of  surgical  tasks. 

For  the  milestone  pertaining  to  the  exploration  of  the  application  of  technologies  to  refine 
methods  of  medical  instruction,  the  following  activities  took  place: 

•  Online  training  system:  we  have  developed  offline  resources  for  training 
laparoscopic  cholesystectomy  procedures  involving  the  use  of  video  vignettes. 
These  resources  are  slated  for  implementation  using  an  online  learning  system. 
We  have  identified  a  doctoral  candidate  in  computer  science  to  facilitate 
implementation  of  the  system. 

•  Audience  polling  system:  We  have  recently  acquired  a  system  for  audience- 
response  measurement.  Our  system  for  audience-response  measurement  is  now 
integrated  into  the  training  of  all  residents  for  polling  response  to  training  and 
medical  grand  rounds.  This  system  will  also  be  used  to  facilitate  and  enhance  our 
presentation  at  the  annual  meeting  of  the  Society  for  Simulation  in  Healthcare. 

•  Simulation-based  competency  based  training:  We  are  currently  refining  and 
expanding  our  use  of  criteria-based  training,  which  uses  measures  of  performance 
to  determine  training  sufficiency.  We  are  currently  refining  the  criteria  for  Virtual 
reality  (VR)  and  physical-model  training  for  basic  laparoscopic  skills. 

For  the  milestone  related  to  developing  new  models  for  simulation  training: 

•  Arthroscopic  simulation:  Arthroscopic  skills  models  continue  to  be  refined.  A 
new  model  for  spinal  disk  herniation  has  been  developed  in  the  fonn  of  a 
prototype.  Pilot  testing  of  the  model’s  functions  has  been  carried  out.  Curriculum 
for  the  model  is  being  developed. 
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•  Ventral  hernia:  Provisional  patent  obtained.  Prosecution  of  full  patent  is  in 
process.  Pilot  validation  trials  of  our  model  have  been  carried  out  by  another 
university.  The  simulator  is  now  in  routine  use  as  a  teaching  tool  for  fellows, 
residents,  and  industry  representatives,  and  provides  a  cost-effective  alternative  to 
porcine  model  of  ventral  hernia  repair.  Validation  trials  are  being  designed. 

•  Suture-skills  drill:  A  new  physical  model  of  tissue  and  pattern  of  visual  targets 
marked  upon  the  tissue  was  developed  for  a  flexible  curriculum  of  training 
suturing  skills.  We  are  in  the  process  of  refining  models  for  developing  the 
curriculum  for  drills  to  practice  suturing  skills. 

For  the  milestone  related  to  configuring  the  training  site  for  OR,  ICU,  Emergency 
Department  and  Team  Training  scenarios: 

New  equipment  and  infrastructure  were  acquired  during  this  reporting  period. 
Specifically,  additional  simulation  equipment  to  be  used  for  a  variety  of 
procedural  and  skill-based  instruction  has  been  delivered.  The  Sim  baby 
simulator  has  been  obtained,  a  pediatric  exam  bed  has  been  integrated  into  the 
simulation  space,  ultrasound  equipment  was  obtained,  allowing  ultrasound-guided 
catheter  placement  training,  curtains  and  storage  have  been  installed,  and  new 
part-task  trainers  have  been  obtained. 

•  Significant  progress  has  been  made  in  the  configuration  of  and  use  of  OR  B  as  a 
training  site.  Video  and  audio  cabling  and  networking  hardware  has  been 
installed  and  all  communication  systems  are  near  completion. 

•  Hiring  of  a  simulation  educator/technician  has  brought  us  to  near  full  operation  of 
the  OR  B  training  site. 

•  Many  new  full  and  part-task  trainers  have  allowed  the  beginning  of  several 
courses  with  a  large  number  waiting  in  the  wings.  More  than  a  dozen  new 
courses  for  medical  and  surgical  residents,  nursing  personnel  and  medical 
students  have  been  introduced  using  this  new  training  site. 

For  the  development  of  collaborative  ventures  with  academic  institutions  and  government 
agencies,  we  are  currently  preparing  extramural  grant  proposals  on  the  topic  of 
developing  metrics  for  surgical  training,  specifically  regarding  the  assessment  of  medical 
resident  perfonnance.  Substantive  collaboration  is  dependent  on  receipt  of  funding. 


C.  Smart  Image 

C.l  Smart  Image:  CT-guided  Imaging 

Having  completed  the  entirety  of  the  experimental  and  animal  imaging  work,  our  efforts 
remained  focused  on  data  processing  and  scientific  reporting  during  the  current  year.  The 
following  were  significant  notable  outcomes. 

1 .  We  made  an  oral  presentation  on  our  work  on  Live  AR  at  the  SAGES  conference  help 
in  Phoenix,  AZ  in  April  2009.  Subsequently,  an  in-depth  manuscript  on  the  same  topic 
was  submitted  to  Surgical  Endoscopy,  the  official  journal  of  SAGES,  for  possible 
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publication.  The  submitted  manuscript  is  placed  in  the  Appendix.  This  reporting 
period  included  work  on  refining  and  producing  results,  especially  Live  AR  movie  clips, 
for  the  conference  presentation  and  the  manuscript. 

2.  We  also  presented  a  poster  at  the  Computer- Assisted  Radiology  and  Surgery  (CARS) 
conference  in  Berlin,  Germany  in  June  2009.  This  presentation  explored  the  specific  topic 
of  using  image  registration  for  continuous  volumetric  CT-guided  interventions.  A  copy  of 
the  abstract  is  presented  in  the  Appendix.  A  manuscript  will  follow  this  presentation. 

3.  We  continued  to  work  on  low-dose  CT  reconstruction,  one  of  the  originally  proposed 
technical  objectives.  We  worked  with  Philips,  the  manufacturer  of  our  CT  scanner,  on 
data  preprocessing  issues  that  now  enable  us  to  reconstruct  images  with  a  consistent 
orientation  and  thereby  allow  head-to-head  comparison  of  reconstructed  images  when  x- 
ray  dose  is  varied.  A  manuscript  summarizing  the  work  will  be  prepared  and  submitted  in 
the  upcoming  year. 


C.2.  Smart  Image:  Image  Pipeline 

The  work  we  have  completed  in  the  past  year  is  summarized  in  a  number  of  peer- 
reviewed  publications  and  has  been  shared  in  several  presentations.  These  are  discussed 
in  the  context  of  our  overall  goals  -  to  develop  visualization  requirements,  principles,  and 
frameworks,  as  well  as  solutions  to  specific  computational  challenges,  which  will  permit 
useful  and  usable  augmentation  of  the  laparoscopic  image.  This  augmentation  assumes 
input  from  other  modalities,  including  surgeons’  annotations,  as  well  as  pre-operative  and 
intra-operative  CT  and  other  imaging  techniques.  The  full  papers  are  in  the  appendix. 

Visualization  Framework: 

We  have  described  the  overall  visualization  framework  for  the  Smart  Image  project, 
including  both  the  computational  and  usability  components,  in  a  paper  submitted  to 
Surgical  Innovations'. 

1)  Yang,  R.}  Carswell,  C.M.,  Wang,  X.,  Zhang,  Q.,  Han,  Q.,  Lio,  C. ,  and  Seales,  B. 
Mapping  the  Way  to  a  Dual  Display  Framework  for  Laparoscopic  Surgery. 

Abstract: 

Many  performance  and  workload  problems  associated  with  the  use  of  traditional 
laparoscopic  displays  are  the  result  of  spatial  disorientation.  This  premise  has  guided 
our  development  of  a  dual  display  framework  for  computer-augmented  surgical  displays, 
allowing  us  to  take  guidance  from  research  on  how  to  design  successful  navigation  aids 
(navaids)  for  large-scale  environments.  Our  dual-display  combines  the  traditional  scope 
(forward  track)  view  with  a  computationally-generated  global  3D  (map)  view.  The  latter 
provides  a  wider  field  of  view,  explicit  cues  to  depth  and  scale,  and  a  way  to  view  interior 
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and  exterior  surfaces  of  target  anatomy  from  different  approach  angles.  One  way  to 
implement  such  a  3D  view  is  to  extract  images  of  surface  textures  from  a  laparoscopy 
video  sequence  and  then  map  the  texture  onto  pre-built  3D  objects,  for  example  surface 
models  derived  from  MRJCT.  We  describe  an  algorithm  that  takes  advantage  of  the  fact 
that  nearby  frames  within  a  video  sequence  usually  contain  enough  coherence  to  allow 
2D-2D  registration,  a  much  better  understood  problem  than  2D-3D  registration.  Our 
texturing  process  can  be  bootstrapped  by  an  initial  2D-3D  manual-assisted  registration 
of  the  first  video  frame  followed  by  mostly -automatic  texturing  of  subsequence  frames. 
Initial  research  on  the  validity  of  our  technical  approach  indicates  that  it  improves 
registration  performance  compared  to  a  standard  registration  technique  that  relies  on 
camera  tracking.  Ongoing  technical  and  usability  evaluations  of  the  system  are  being 
conducted  in  order  to  ensure  system  functionality. 

Front  end  assessments  for  requirements  and  acceptability  are  described  in  a  peer- 
reviewed  proceedings  paper  presented  at  the  2009  Human  Factors  and  Ergonomics 
Society  meeting. 


2)  Lio,  C.H.,  Carswell,  C.M.,  Han,  Q.,  Park,  A.,  Strup,  S.,  Selaes,  W.B.,  Clarke,  D.,  Lee, 
G.,  and  Hoskins,  J.  (2009).  Using  Formal  Qualitative  Methods  to  Guide  Early 
Development 

of  an  Augmented  Reality  Display  System  for  Surgery.  Proceedings  of  the  Human  Factors 
and  Ergonomics  Society  53rd  Annual  Meeting.  Santa  Monica,  CA:  HFES. 

Abstract: 


Nine  laparoscopic  surgical  experts  (2  residents,  4  fellows,  and  3  surgeons)  underwent 
semi-structured  interview  questions  to  evaluate  the  concept  of  a  “ dual-view ”  display  for 
laparoscopic  surgery.  The  30-40  minute  audio-recorded  interviews  were  transcribed, 
submitted  to  an  open  source  qualitative  program  for  classification  and  categorizing,  and 
were  condensed  for  the  iterative  processes  of  analysis  and  interpretation.  Findings 
revealed  that  despite  the  relatively  brief  interview  sessions  and  limited  number  of 
surgical  experts  available,  the  experts  provided  sufficient  insights  and  suggestions  to 
guide  further  development  of prototypes.  This  means  that  the  use  of  semi-structured 
interviews  as  an  expert  knowledge  elicitation  technique  may  be  suitable  for  assessing  the 
development  of  augmented  reality  display  systems  for  surgical  and  training  applications, 
and  it  may  have  promise  for  the  development  of  augmented  and 
virtual  environments  more  genially. 

Computational  Challenges: 

The  past  year  also  saw  the  publication  of  a  peer-reviewed  journal  article  summarizing  the 
procedure  we  have  developed  for  registering  a  series  of  video  images  from  the 
laparoscope  to  prebuilt  surface  models  without  using  camera  tracking. 

3)  Want,  X.,  Zhang,  Q.,  Han,  Q.,  Yang,  R.,  Carswell,  M.,  Seales,  B.,  and  Sutton,  E. 

(2009)  Endoscopic  video  texture  mapping  on  pre-built  3D  anatomical  objects  without 
camera  tracking.  IEEE  Transactions  on  Medication  Imaging,  7(7),  1-12. 
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Abstract: 


Traditional  minimally  invasive  surgeries  use  a  view  port  provided  by  an  endoscope  or 
laparoscope.  We  argue  that  a  useful  addition  to  typical  endoscopic  imagery  would  be  a 
global  3D  view  providing  a  wider  field  of  view  with  explicit  depth  information  for  both 
the  exterior  and  interior  of  target  anatomy.  One  technical  challenge  of  implementing 
such  a  view  is 

finding  efficient  and  accurate  means  of  registering  texture  images  from  the  laparoscope 
on  pre-built  3D  surface  models  of  target  anatomy  derived  from  magnetic  resonance  (MR) 
or  computed 

tomography  (CT)  images.  This  paper  presents  a  novel  method  for  addressing  this 
challenge  that  differs  from  previous  approaches,  which  depend  on  tracking  the  position 
of  the  laparoscope.  We 

take  advantage  of  the  fact  that  neighboring  frames  within  a  video  sequence  usually 
contain  enough  coherence  to  allow  a  2D-2D  registration,  which  is  a  much  more  tractable 
problem. 

The  texturing  process  can  be  bootstrapped  by  an  initial  2D-3D  user-assisted  registration 
of  the  first  video  frame  followed  by  mostly -automatic  texturing  of  subsequent  frames.  We 
perform 

experiments  on  phantom  and  real  data,  validate  the  algorithm  against  the  ground  truth, 
and  compare  it  with  the  traditional  tracking  method  by  simulations.  Experiments  show 
that  our 

method  improves  registration  performance  compared  to  the  traditional  tracking 
approach. 

We  also  published  a  peer-reviewed  proceedings  paper  on  a  method  for  acquiring  and 
reconstructing  3D  surface  models  based  on  light  fall-off  between  the  camera  and  organ 
surface. 

4)  Liao,  M.,  Wang,  L.,  Yang,  R.,  and  Gong,  M.  Real-time  light  fall-off  stereo.  (2008). 
International  Conference  on  Image  Processing  (ICIP). 

Abstract: 

We  present  a  real-time  depth  recovery  system  using  Light  Fall-off  Stereo  (LFS).  Our 
system  contains  two  co-axial  point  light  sources  (LEDs)  synchronized  with  a  video 
camera.  The 

video  camera  captures  the  scene  under  these  two  LEDs  in  complementary  states(e.g.,  one 
on,  one  off).  Based  on  the  inverse  square  law  for  light  intensity,  the  depth  can  be  directly 
solved  using  the  pixel  ratio  from  two  consecutive  frames.  We  demonstrate  the 
effectiveness  of  our  approach  with  a  number  of  real  world  scenes.  Quantitative 
evaluation  shows  that  our 

system  compares  favorably  to  other  commercial  real-time  3D  range  sensors,  particularly 
in  textured  areas.  We  believe  our  system  offers  a  low-cost  high-resolution  alternative  for 
depth 
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sensing  under  controlled  lighting. 

Yet  another  peer-reviewed  proceedings  paper  was  recently  presented  at  MICAI  (Medical 
Image  Computing  &  Computer  Assisted  Intervention)  and  dealt  with  the  methods  to 
model  the  intra-object  deformations  with  using  a  small  number  of  parameters  that  can  be 
applied  to  new  target  objects.  This  allows  for  better  registration  of  pre-built  3D  shape 
models  of  target  organs  to  their  corresponding  laparoscopic  video  sequence. 

5)  Han,  Q.  Strap,  S.,  Carswell,  C.M.,  Clarke,  D.,  Seales,  W.B.  (2009).  Model 
Completion  via  Deformation  Cloning  Based  on  an  Explicit  Global  Deformation  Model. 
12th  International  Conference  on  Medical  Image  Computing  &  Computer  Assisted 
Intervention  (MICCAI)(1)  2009:  1067-1074. 

Abstract: 

Our  main  focus  is  the  registration  and  visualization  of  a  pre-built  3D  model  from 
preoperative  images  to  the  camera  view  of  a  minimally  invasive  surgery  (MIS).  Accurate 
estimation  of  soft-tissue  deformations  is  key  to  the  success  of  such  a  registration.  This 
paper  proposes  an  explicit  statistical  model  to  represent  global  non-rigid  deformations. 
The  deformation  model  built  from  a  reference  object  is  cloned  to  a  target  object  to  guide 
the  registration  of  the  pre-built  model,  which  completes  the  deformed  target  object  when 
only  a  part  of  the  object  is  naturally  visible  in  the  camera  view.  The  registered  target 
model  is  then  used  to  estimate  deformations  of  its  substructures.  Our  method  requires  a 
small  number  of  landmarks  to  be  reconstructed  from  the  camera  view.  The  registration  is 
driven  by  a  small  set  of parameters,  making  it  suitable  for  real-time  visualization. 

Continuing  Work: 

We  shipped  the  completed  dual  display  simulation  (interactive  prototype  for  usability 
testing)  to  UMMC.  This  allows  for  the  precise  assessment  of  the  presumed  advantages 
of  our  dual  display  framework  for  navigation  and  the  development  of  adequate  spatial 
situation  models.  It  is  our  hope  that  the  framework  will  fonn  the  basis  for  continued 
collaboration  and  development  of  user-centered  visualization  systems  for  minimally 
invasive  surgery. 

Key  Research  Accomplishments 

A.  Informatics 

Informatics  subgroup  1.  Perioperative  Scheduling  Study 

Major  Accomplishments  achieved  during  this  period  of  performance  include  the 
development  of  a  mathematical  congestion  evaluation  model  for  evaluating 
congestion  in  post-operative  units,  including  ICUs,  IMCs,  and  floor  units.  This  model 
requires  data  about  post-operative  destinations  and  length-of-stay  distributions  for 
different  types  of  surgeries.  We  analyzed  data  about  cardiac  surgeries  from  two 
years  and  have  analyzed  UMMC  financial  records  for  all  of  the  surgical  cases  for  a  year. 
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Informatics  subgroup  2.  Operating  Room  Glitch  Analysis 

A  tactical  dashboard  was  developed  in  conjunction  with  the  strategic  dashboard  and 
has  been  deployed  to  an  internally  hosted  server.  The  tactical  dashboard  is  being  used 
within  the  perioperative  environment  to  evaluate  OR  utilization,  scheduling  workflow, 
and  case  data  accuracy.  Data  are  being  validated  at  the  case  level  and  new  processes  for 
data  entry  are  being  designed  to  ensure  the  accuracy  of  the  metrics  within  the  strategic 
dashboard. 

Informatics  subgroup  3.  Context  Aware  Surgical  Training  (CAST) 

A  prototype  CAST  system  was  emplaced  in  the  MASTRI  system  for  assessment. 
Work  was  done  to  design  a  system  of  evaluation  of  the  system  in  terms  of  improvements 
in  learning  outcomes  due  to  self-feedback,  improvements  in  learning  outcomes  due  to 
instructor  feedback  and  synchronous  versus  asynchronous  feedback.  We  demonstrated 
the  system  to  a  set  of  resident  volunteers  for  feedback  in  a  form  of  Beta-test  of  the 
system.  We  set  up  hardware  and  software  to  include  the  VR  Simulator  as  part  of  the 
CAST  system  deployment.  We  got  usability  feedback  and  fixed  bugs. 

Informatics  subgroup  3.  Video  Summarization 

We  completed  an  initial  analysis  of  this  problem.  We  compared  more  than  50  image 
features  with  a  distance  metric  to  identify  the  critical  view  of  a  laparoscopic 
cholecystectomy.  We  experimented  with  roughly  50  different  images  features  and  several 
distance  metrics.  Our  initial  results  showed  a  72%  sensitivity  and  72%  specificity.  An 
abstract  was  submitted  to  the  American  Medical  Informatics  Association  (AMIA)  Fall 
Symposium. 

Informatics  subgroup  4.  Operating  Room  Clutter  (ORC) 

During  this  period  of  perfonnance,  we  published  8  full-length  peer  reviewed  journal 
articles,  2  full-length  peer  reviewed  proceeding  articles,  and  8  conference  abstracts. 

B.  Simulation  (Virtual  Patient) 

n  this  final  year  of  the  project,  our  team  has  delivered  additional  versions  of  the 
Maryland  Virtual  Patient  Environment.  The  realism  of  the  simulation  has  been  enhanced 
by  including  coverage  of  “unexpected”  interventions;  allowing  discontinued  treatments; 
allowing  new  diseases  to  develop  due  to  side  effects  of  treatments.  The  user  interface  has 
been  redesigned.  A  new  agent-based  architecture  has  been  developed  to  support  enhanced 
cognitive  capabilities  of  the  virtual  patient  and  the  intelligent  tutor,  including  language 
capabilities.  In  the  area  of  language  processing,  a  dialog  processing  model  was 
developed.  Work  has  continued  on  improving  the  language  understanding  capabilities, 
centrally  including  treatment  of  referring  expressions.  Enhancement  of  static  knowledge 
resources,  the  ontology  and  the  lexicon,  has  been  ongoing.  Work  on  extending  the 
coverage  of  diseases  has  been  ongoing:  a  further  improvement  of  the  model  of  GERD  is 
under  way,  as  is  the  modeling  of  cardiovascular  diseases.  A  totally  reworked  system 
version,  with  dialog  support,  was  released  in  June  2008.  Work  has  also  been  ongoing  on 
improving  and  extending  the  set  of  development  tools  -  the  DEKADE  demonstration, 
evaluation  and  knowledge  acquisition  environment  supporting  natural  language  work  has 
been  revamped;  the  interface  for  creating  instances  of  virtual  patients  has  also  been 
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enhanced;  a  web-based  environment  for  supporting  internal  documentation  has  been 
installed. 


B.  Simulation  (Training  for  Surgical  Excellence) 

For  the  milestone  related  to  configuring  the  training  site  for  OR,  ICU,  Emergency 
Department  and  Team  Training  scenarios,  new  equipment  and  infrastructure  were 
acquired  during  this  reporting  period.  Specifically,  additional  simulation  equipment  to 
be  used  for  a  variety  of  procedural  and  skill-based  instruction  has  been  delivered.  The 
Sim  baby  simulator  has  been  obtained,  a  pediatric  exam  bed  has  been  integrated  into  the 
simulation  space,  ultrasound  equipment  was  obtained,  allowing  ultrasound-guided 
catheter  placement  training,  curtains  and  storage  have  been  installed,  and  new  part-task 
trainers  have  been  obtained.  Significant  progress  has  been  made  in  the  configuration  of 
and  use  of  OR  B  as  a  training  site.  Video  and  audio  cabling  and  networking  hardware 
has  been  installed  and  all  communication  systems  are  near  completion.  Hiring  of  a 
simulation  educator/technician  has  brought  us  to  near  full  operation  of  the  OR  B  training 
site.  Many  new  full  and  part-task  trainers  have  allowed  the  beginning  of  several  courses 
with  a  large  number  waiting  in  the  wings.  More  than  a  dozen  new  courses  for  medical 
and  surgical  residents,  nursing  personnel  and  medical  students  have  been  introduced 
using  this  new  training  site. 

C.  Smart  Image 

C.l.  Smart  Image:  CT  guided  imaging 

We  made  an  oral  presentation  on  our  work  on  Live  AR  at  the  SAGES  conference 
help  in  Phoenix,  AZ  in  April  2009.  Subsequently,  an  in-depth  manuscript  on  the  same 
topic  was  submitted  to  Surgical  Endoscopy,  the  official  journal  of  SAGES.  We  also 
presented  a  poster  at  Computer-Assisted  Radiology  and  Surgery  (CARS)  conference  in 
Berlin,  Gennany  in  June  2009.  This  presentation  explored  the  specific  topic  of  using 
image  registration  for  continuous  volumetric  CT-guided  interventions.  We  continued  to 
work  on  low-dose  CT  reconstruction,  one  of  the  originally  proposed  technical  objectives. 
We  worked  with  Philips,  the  manufacturer  of  our  CT  scanner,  on  data  preprocessing 
issues  that  now  enable  us  to  reconstruct  images  with  a  consistent  orientation  and  thereby 
allow  head-to-head  comparison  of  reconstructed  images  when  x-ray  dose  is  varied.  A 
manuscript  summarizing  the  work  will  be  prepared  and  submitted  in  the  upcoming  year. 

C.2.  Smart  Image:  Image  Pipeline 

The  work  we  have  completed  in  the  past  year  is  summarized  in  a  number  of  peer- 
reviewed  publications  and  has  been  shared  in  several  presentations.  These  are  discussed 
in  the  context  of  our  overall  goals  -  to  develop  visualization  requirements,  principles,  and 
frameworks,  as  well  as  solutions  to  specific  computational  challenges,  which  will  permit 
useful  and  usable  augmentation  of  the  laparoscopic  image.  This  augmentation  assumes 
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input  from  other  modalities,  including  surgeons’  annotations,  as  well  as  pre-operative  and 
intra-operative  CT  and  other  imaging  techniques. 


Reportable  Outcomes 

We  advanced  the  body  of  knowledge  pertaining  to  infonnatics,  smart  image,  simulation 
and  human  factors  as  these  relate  to  surgical  procedures,  the  perioperative  environment 
and  the  training  of  surgery.  We  published  more  than  forty  manuscripts,  hosted  national 
and  international  meetings  related  to  innovation  in  the  surgical  environment,  and 
incorporated  technical  advances  into  patient  care  in  a  large  academic  medical  center.  We 
influenced  significantly  the  training  of  more  than  three  hundred  fellows  and  residents, 
hundreds  of  staff  and  care  providers  and  numerous  medical  students. 

Perhaps  our  most  important  accomplishment  has  been  the  identification  of  a  new  of  basic 
surgical  sciences.  These  include  computer  and  physical  sciences,  infonnatics,  smart 
imaging,  simulation  and  ergonomics  and  human  factors  that  underpin  surgical  training. 
This  event  is  a  landmark  of  sorts,  as  it  has  changed  forever  the  course  of  surgical 
education.  Lessons  learned  from  this  research  effort  are  being  applied  in  training 
programs  throughout  the  country  and  internationally. 


Conclusion 

This  report  began  with  the  recognition  that  an  extraordinary  evolution  in  surgical  care  has 
occurred  caused  by  rapid  advances  in  technology  and  creative  approaches  to  medicine. 
The  increased  speed  and  power  of  computer  applications,  the  rise  of  visualization 
technologies  related  to  imaging  and  image  guidance,  improvement  in  simulation-based 
technologies  (tissue  properties,  tool-tissue  interaction,  graphics,  haptics,  etc)  have 
interacted  to  advance  the  practice  of  surgery.  However,  the  medical  profession  lags 
behind  other  applications  of  infonnation  systems.  The  research  program  reported  here 
has  proceeded  under  the  mantle  of  “Operating  Room  of  the  Future”.  As  a  natural 
occurrence  in  the  outcome  of  lessons  learned  in  medicine,  we  are  replacing  that  theme 
with  the  more  appropriate  “Innovations  in  the  Surgical  Environment.” 

This  research  program  has  consisted  of  three  major  pillars;  OR  informatics,  simulation, 
and  smart  image.  This  year,  we  incorporated  the  research  focus  areas  of  physical  and 
cognitive  ergonomics  and  human  factors  into  the  informatics  pillar.  A  summary 
description  of  the  entire  research  portfolio  was  included  in  the  appendix. 

\ 

The  purpose  of  the  OR  infonnatics  program  is  to  develop,  test,  and  deploy  technologies 
to  collect  real-time  data  about  key  tasks  and  process  elements  in  clinical  operating  rooms. 
We  have  established  testbeds  of  activities  in  both  simulated  and  operational 
environments.  We  are  currently  perfonning  tests  of  the  hardware,  refining  software,  and 
applying  lessons  learned  to  hospital  operational  functions.  The  objective  of  Simulation 
research  is  to  create  a  system  where  a  user  can  interact  with  a  virtual  human  model  in 
cognitive  simulation  and  have  the  virtual  human  respond  appropriately  to  user  queries 
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and  interventions  in  clinical  situations,  with  a  focus  on  cognitive  decision  making  and 
judgment.  We  have  made  significant  strides  toward  realizing  these  goals.  The  MVP 
simulation  functions  well  for  esophageal  disorders,  and  is  continuing  to  expand  the 
repertoire  of  diseases  that  are  in  the  simulation  model. 

The  objective  of  smart  image  is  use  real-time  3D  ultrasonography  and  40-slice 
highframe-rate  computed  tomography  (CT)  for  intraoperative  imaging  to  volume 
rendered  anatomy  from  the  perspective  of  the  endoscope.  We  are  combining  CT  and 
Ultrasound  to  overlay  image  and  data  to  enhance  the  perfonnance  of  surgeons-in- 
training.  We  have  carried  out  animate  model  testing  of  the  image  registration  with  great 
success.  We  continue  to  refine  and  expand  our  capability  through  hardware  and  software 
refinement. 

In  the  future,  OR  workspace  layout  would  be  optimized  through  ergonomic  data  and 
human  factors  analysis,  and  this  optimization  would  lead  to  the  establishment  of  “best 
practices”  for  an  array  of  surgical  operations.  Proper  layout  would  reduce  risks  of 
infection,  speed  operations,  and  reduce  fatigue  of  surgeons  and  staff,  all  elements  that 
could  contribute  to  a  reduction  in  AEs  and  improved  patient  safety. 

The  year  ahead  is  full  of  promise  for  refinements  in  the  use  of  infonnatics  to  support  safe 
and  efficient  operating  room  procedures,  the  use  of  simulation  to  improve  and  accelerate 
the  training  of  competent  surgeons,  and  the  blending  of  imaging  capabilities  to  provide 
clearer  and  safer  interactions  between  patient  and  surgeon. 

As  stated  earlier  in  this  report,  the  current  contract,  W81XWH-06-2-0057,  has  been  tied 
to  a  prior  and  topically  related  contract,  DAMD- 17-03-2-0001.  The  prior  contract  closed 
in  February  of  2009;  the  current  one  was  scheduled  to  close  in  October  of  2009;  this 
contract  was  granted  a  no-cost  extension  until  October,  2010.  Some  projects  contained  in 
the  Informatics  pillar,  OGA,  ORC  and  IPC,  have  been  completed.  The  WORQ  project 
will  continue  under  the  current  contract  as  will  the  CAST  project  that  has  been  reshaped 
into  Video  Summarization.  Simulation  for  Training  and  the  ergonomics/human  factors 
work  will  continue  through  the  period  of  no-cost  extension. 

These  changes  represent  the  maturing  of  a  research  endeavor  over  the  course  of  six  years, 
an  endeavor  which  opened  the  door  to  a  new  set  of  basic  surgical  sciences.  The 
Innovations  in  the  Surgical  Environment  conference  planned  for  the  spring  of  2010  will 
summarize  the  entirety  of  the  research  effort  and  point  the  direction  to  future  innovative 
approaches  to  advance  surgical  technology  in  behalf  of  patient  safety. 


Publications 

Publications  Associated  with  the  Conduct  of  the  Research 

(2006) 

■  Agarwal,  S.,  Joshi,  A.,  Finin,  T.  and  Yesha,  Y.  A  Pervasive  Computing  System 

■  for  the  Operating  Room  of  the  Future,  technical  report  TR-CS-06-xx,  Computer 


28 


■  Science  and  Electrical  Engineering,  University  of  Maryland,  Baltimore  County, 

■  September  2006.  http://ebiquity.umbc.edu/paper/html/id/339/ 

■  Agarwal,  S.,  Context- Aware  System  to  Create  Electronic  Medical  Encounter 

■  Records,  M.S.  thesis,  Computer  Science  and  Electrical  Engineering,  University 
of 

■  Maryland,  Baltimore  County,  May  2006. 

■  http  ://ebiquity.umbc .  edu/paper/html/id/3  3  8/ 

■  Agarwal,  S.,  Joshi,  A.,  Finin,  T.,  Ganous,  T.  and  Yesha,  Y.  Context-Aware 

■  System  to  Create  Electronic  Medical  Records,  technical  report  TR-CS-06-05, 

■  Computer  Science  and  Electrical  Engineering,  University  of  Maryland, 

Baltimore 

■  County,  July  2006.  http://ebiquity.umbc.edU/paper/html/id/3 12/ 

■  Akshay  Java,  Tim  Finin  and  Sergei  Nirenburg,  Text  understanding  agents  and 
the  Semantic  Web,  39th  Hawaii  Int.  Conf.  on  System  Sciences,  Kauai  HI,  4-6 
January,  2006. 

■  Akshay  Java,  Tim  Finin,  and  Sergei  Nirenburg,  SemNews:  A  Semantic  News 
Framework,  Proceedings  of  the  21st  National  Conf.  on  Artificial  Intelligence, 
July  2006. 

■  Anand  Patwardhan,  Filip  Perich,  Anupam  Joshi,  Tim  Finin  and  Yelena  Yesha, 
Querying  in  packs:  Trustworthy  Data  Management  in  Ad  Hoc  Networks, 
International  Journal  of  Wireless  Information  Networks,  2006. 

■  Boanerges  Aleman-Meza,  Meenakshi  Nagarajan,  Cartic  Ramakrishnan,  Amit 
Sheth,  Budak  Arpinar,  Li  Ding,  Pranam  Kolari,  Anupam  Joshi,  and  Tim  Finin, 
"Semantic  Analytics  on  Social  Networks:  Experiences  in  Addressing  the 
Problem  of  Conflict  of  Interest  Detection",  WWW2006,  Edinburgh,  May  2006. 

■  Cynthia  Parr,  Andriy  Parafiynyk,  Joel  Sachs,  Li  Ding,  Sandor  Dornbush,  Tim 
Finin,  Taowei  Wang,  and  Allan  Hollander,  Integrating  Ecoinformatics  Resources 
on  the  Semantic  Web,  poster  paper,  15th  International  World  Wide  Web  Conf., 
Edinburgh,  May  2006. 

■  Dandekar  O,  Walimbe  V,  Siddiqui  K,  Shekhar  R:  Image  registration  accuracy 

■  with  low-dose  CT :  How  low  can  we  go?  In  Proceedings  of  2006  IEEE 

■  International  Symposium  on  Biomedical  Imaging,  p  502-505. 

■  Dandekar  O,  Walimbe  V,  Shekhar  R:  Hardware  implementation  of  hierarchical 

■  volume  subdivision-based  elastic  registration.  In  Proceedings  of  the  28th  Annual 

■  International  Conference  of  the  IEEE  Engineering  in  Medicine  and  Biology 

■  Society  (IEEE  EMBC  2006),  p  1425-1428. 

■  Dandekar  O,  Castro-Pareja  CR,  Shekhar  R,"FPGA -based  Reconfigurable  3D 

■  Image  Pre-processing  for  Image  Guided  Interventions.  Journal  of  Real-Time 

■  Image  Processing.  (Submitted) 

■  Goodman  L,  Gulsun  M,  Washington  L,  Nagy  P,  Piacsek  K  Inherent  Variability 
of  CT  Lung  Nodule  Measurements  In  Vivo  Using  Semi  automated  Volumetric 
Measurements  Am  J  Roentgenol,  186:  989-994,  2006 

■  Hu  P,  Xiao  Y,  Ho  D,  Mackenzie  CF,  Hu  H,  Voigt  R,  Martz  D.  Advanced 

■  Visualization  Platfonn  for  Surgical  Operating  Room  Coordination:  Distributed 

■  Video  Board  System.  Surgical  Innovation.  13(2):  129-135.  2006 

■  Jarrell  B,  Nirenburg  S,  McShane  M,  Fantry  G,  Beale  S,  Mallott,  D,  Raczek  J: 


29 


■  “Simulation  for  Teaching  Decision  Making  in  Medicine:  The  Next  Step”,  In 

■  Proceedings  of  Medicine  Meets  Virtual  Reality  :  IGT-Registration  &  Navigation, 

■  2007. 

■  Jim  Parker,  Anand  Patwardhan,  Filip  Perich,  Anupam  Joshi  and  Tim  Finin,  Trust 
in  Pervasive  Computing,  in  The  Handbook  of  Mobile  Middleware,  P.  Bellavista 
and  A.  Corradi  (eds.),  pp.  473-496,  CRC  Press,  May  2006. 

■  Kavic  SM,  Segan  RD,  George  IM,  Turner  PL,  Roth  JS  and  Park  A.  Classification 
of  hiatal  hernias  using  dynamic  three-dimensional  reconstruction.  Surg 
Innovation.  2006;  13:  21-25. 

■  Kavic  SM,  Segan  RD,  George  IM,  Turner  PL,  Roth  JS,  Park  AE.  Classification 
of  hiatal  hernias  using  dynamic  three-dimensional  reconstruction,  Surg  Innov 
13(1):  49-52,  2006 

■  Kavic  SM,  Segan  RD,  George  IM,  Turner  PL,  Roth  JS,  Park  AE.  Classification 
of  hiatal  hernias  using  dynamic  three-dimensional  reconstruction,  Surg  Innov, 
March  2006. 

■  Lalana  Kagal  and  Tim  Finin,  Modeling  Conversation  Policies  using  Permissions 
and  Obligations,  Journal  of  Autonomous  Agents  and  Multi-Agent  Systems,  to 
appear,  2006. 

■  Nagy  P,  George  I,  Bernstein  W,  Caban  J,  Klein  R,  Mezrich  R,  and  Park  A.  Radio 
frequency  identification  systems  technology  in  the  surgical  setting.  Surg  Innov 
13(1):  61-67,2006 

■  Nagy  P,  George  I,  Bernstein  W,  Caban  J,  Klein  R,  Mezrich  R,  Park  A.  Radio 
frequency  identification  systems  technology  in  the  surgical  setting.  Surg  Innov 
13(1):  61-67,2006 

■  Nirenburg,  S.,  M.  McShane,  S.  Beale,  T.  O’Hara,  G.  Fantry,  J.  Raczek,  B.  Jarrell, 
“Cognitive  Simulation  in  Virtual  Patients”,  accepted  for  presentation  at  the  19th 
International  FLAIRS  Conference,  Melbourne  Beach,  Florida,  May  11-13,  2006. 

■  Pranam  Kolari  and  Tim  Finin,  A  Framework  for  Multi-Relational  Analytics  on 
the  Blogosphere,  (abstract),  Proceedings  of  the  21st  National  Conf.  on  Artificial 
Intelligence,  July  2006. 

■  Pranam  Kolari,  Akshay  Java  and  Tim  Finin,  "Characterizing  the  Splogosphere", 
Third  Annual  Workshop  on  Weblogging  Ecosystem:  Aggregation,  Analysis  and 
Dynamics,  held  in  conjunction  with  WWW2006,  Edinburgh,  May  2006. 

■  Pranam  Kolari,  Akshay  Java,  Tim  Finin,  Tim  Oates,  and  Anupam  Joshi, 
Detecting  Spam  Blogs:  A  Machine  Learning  Approach,  Proceedings  of  the  21st 
National  Conf.  on  Artificial  Intelligence,  July  2006. 

■  Pranam  Kolari,  Tim  Finin  and  Anupam  Joshi,  SVMs  for  the  Blogosphere:  Blog 
Identification  and  Spam  Detection,  AAAI  Spring  Symposium  on  Computational 
Approaches  to  Analyzing  Weblogs,  Stanford,  March  2006. 

■  Pranam  Kolari,  Tim  Finin,  Yelena  Yesha,  Kelly  Lyons,  Jen  Hawkins  and 
Stephen  Perelgut,  Policy  Management  of  Enterprise  Systems:  A  Requirements 
Study,  short  paper,  2006  IEEE  Workshop  on  Policy  for  Distributed  Systems  and 
Networks,  5-7  June  2006,  London,  Ontario  CN. 

■  Ross,  J.M.,  T.M.  Bayles,  C.  Parker,  S.L.Titus,  B.  Jarrell  and  J.  Raczek, 
Engineering  in  Health  Care  Multimedia  Curriculum  for  High  School  Technology 
Education,  accepted  for  presentation  in  the  K-12  Engineering  Outreach  Division 


30 


of  the  2006  American  Society  for  Engineering  Education  Annual  Conference  & 
Exposition  in  Chicago,  IL,  June  2006 

■  Shekhar  R,  Dandekar  O,  Kavic  S,  George  I,  Mezrich  R,  Park  A,  "Development 
of 

■  continuous  CT-guided  minimally  invasive  surgery,"  In  Proceedings  of  SPIE 

■  Medical  Imaging  2007:  Visualization  and  Image-Guided  Procedures,  vol.  6509, 

■  pp.  65090D,  2007 

■  Shekhar  R,  Dandekar  O,  Kavic  S,  George  I,  Mezrich  R,  Park  A,  "Development 
of 

■  16  continuous  CT-guided  minimally  invasive  surgery,"  In  Proceedings  of 
Medicine 

■  Meets  Virtual  Reality:  IGT-Registration  &  Navigation,  2007 

■  Tim  Finin  and  Li  Ding,  Search  Engines  for  Semantic  Web  Knowledge, 
Proceedings  of  XTech  2006:  Building  Web  2.0,  16-19  May  2006. 

■  Vartak,  N.,  Protecting  the  privacy  of  RFID  tags.  (M.S.  thesis)  Computer  Science 

■  and  Electrical  Engineering,  University  of  Maryland,  Baltimore  County,  May 

■  2006. 

■  Walimbe  V,  Shekhar  R:  Automatic  elastic  image  registration  by  interpolation  of 
3D  rotations  and  translations  from  discrete  rigid-body  transformations,  Medical 
Image  Analysis  of  3D  rotations  and  translations  from  discrete  rigid-body 
transformations,  Medical  Image  Analysis 

■  Wasei  M,  Xiao  Y,  Wieringa  P,  Strader  M  ,  Hu  P,  Mackenzie  C.  Visualization  of 

■  Uncertainty  to  Support  Collaborative  Trajectory  Management  in  Hospital  Care. 

■  2006  Conference  of  International  Ergonomics  Association.372 1  -3726.  2006 

■  35.  Xiao  Y,  Strader  M,  Hu  P,  Wasei  M,  Wieringa  P.  Visualization  Techniques 
for 

■  Collaborative  Trajectory  Management .  ACM  Conference  on  Human  Factors  in 

■  Computing  Systems,  pp.1547  -  1552.  2006 

■  Xiao  Y,  Wasei  M,  Hu  P,  Wieringa  P,  Dexter  F.  Dynamic  Management  in 

■  Perioperative  Processes:  A  Modeling  and  Visualization  Paradigm.  12th  IF  AC 

■  Symposium  on  Information  Control  Problems  in  Manufacturing.  (3)647-52.  2006 

■ 

(2007) 

■  Dexter  F,  Epstein  RH,  Traub  RD,  &  Xiao  Y.  Making  Management  Decisions  on 
the  Day  of  Surgery  Based  on  Operating  Room  Efficiency  and  Patient  Waiting 
Times.  Anesthesiology,  101(6):  1444-1453.  2004 

■  Dexter  F,  Xiao  Y,  Dow  AJ,  Strader  MM,  Ho  D,  Wachtel  RE.  Coordination  of 
Appointments  for  Anesthesia  Care  Outside  of  Operating  Rooms  Using  an 
Enterprise  Wide  Scheduling  System.  Anesthesia  and  Analgesia.  105:1701-1710. 
2007 

■  Dexter  F,  Xiao  Y,  Dow  AJ,  Strader  MM,  Ho  D,  Wachtel  RE.  Coordination  of 
Appointments  for  Anesthesia  Care  Outside  of  Operating  Rooms  Using  an 
Enterprise  Wide  Scheduling  System.  Anesthesia  and  Analgesia.  105:1701-1710. 
2007 

■  Dutton  R,  Ho  D,  Hu  P,  Mackenzie  CF,  Xiao  Y.  Decision  Making  by  Operating 
Room  Managers:  The  Burden  of  Changes.  Anesthesiology,  103  :A1 175.  2005 


31 


■  Dutton  R,  Hu  P,  Seagull  FJ,  Scalea  T,  Xiao  Y, .  Video  for  Operating  Room 
Coordination:  Will  the  Staff  Accept  It?.  Anesthesiology:  101:  A1389.  2004 

■  Dutton  R,  Hu  PF,  Mackenzie  CF,  Seebode  S,  Xiao  Y.  A  Continuous  Video 
Buffering  System  for  Recording  Unscheduled  Medical  Procedures. 
Anesthesiology,  103:A1241.  2005 

■  Gilbert  TB,  Hu  PF,  Martz  DG,  Jacobs  J,  Xiao  Y.  Utilization  of  Status  Monitoring 
Video  for  OR  Management.  Anesthesiology,  103:A1263.  2005 

■  Hu  P,  Seagull  FJ,  Mackenzie  CF,  Seebode  S,  Brooks  T,  XiaoY.  Techniques  for 
Ensuring  Privacy  in  Real-Time  and  Retrospective  Use  of  Video.  Telemedicine 
and  e-Health,  12(2):  204,  T1E1.  2006 

■  Hu  P,  Xiao  Y,  Ho  D,  Mackenzie  CF,  Hu  H,  Voigt  R,  Martz  D.  Advanced 
Visualization  Platfonn  for  Surgical  Operating  Room  Coordination:  Distributed 
Video  Board  System.  Surgical  Innovation.  13(2):  129-135.  2006 

■  Hu  PF,  Burlbaugh  M,  Xiao  Y,  Mackenzie  CF,  Voigt  R,  Brooks  T,  Fraser  L, 
Connolly  MR,  Herring  T.  Video  Infrastructure  and  Application  Design  Methods 
for  an  OR  of  the  Future.  Telemedicine  and  e-Health.  1 1(2),  211,  T3C2.  2005 

■  Hu  PF,  Hu  H,  Seagull  JF,  Mackenzie  CF,  Voigt  R,  Martz  D,  Dutton  R,  Xiao  Y. 
Distributed  Video  Board:  Advanced  Telecommunication  System  for  Opearation 
Room  Coordination.  Telemedicine  and  e-Health.  1 1(2),  248,  P28.  2005 

■  Hu  PF,  Xiao  Y,  Mackenzie  CF,  Seagull  FJ,  Brooks  T,  LaMonte  MP,  &  Gagliano 
D.  Many  to  One  to  Many  Telemedicine  Architecture  and  Applications. 
Telemedicine  Journal  and  e-Health.  10(Supplement  1),  S-39.  2004 

■  Kim  Y-J,  Xiao  Y,  Hu  P,  Dutton  RP.  Staff  Acceptance  of  Video  Monitoring  for 
Coordination:  A  Video  System  to  Support  Perioperative  Situation  Awareness. 
Journal  of  Clinical  Nursing  (accepted).  2007 

■  Kim  Y-J,  Xiao  Y,  Hu  P,  Dutton  RP.  Staff  Acceptance  of  Video  Monitoring  for 
Coordination:  A  Video  System  to  Support  Perioperative  Situation  Awareness. 
Journal  of  Clinical  Nursing  (accepted).  2007 

■  Dandekar  and  R.  Shekhar,  "FPGA-accelerated  Deformable  Registration  for 
Improved  Target-delineation  During  CT-guided  Interventions,"  IEEE 
Transactions  on  Biomedical  Circuits  and  Systems,  vol.  1(2),  pp.  1 16-127,  2007. 

■  Dandekar,  C.  Castro-Pareja,  and  R.  Shekhar,  "FPGA-based  real-time  3D  image 
preprocessing  for  image-guided  medical  interventions,"  Journal  of  Real-Time 
Image  Processing,  vol.  1(4),  pp.  285-301,  2007. 

■  Dandekar,  K.  Siddiqui,  V.  Walimbe,  and  R.  Shekhar,  "Image  registration 
accuracy  with  low-dose  CT:  how  low  can  we  go?,"  in  3rd  IEEE  International 
Symposium  on  Biomedical  Imaging:  Nano  to  Macro,  2006,  pp.  502-505. 

■  Dandekar,  V.  Walimbe,  and  R.  Shekhar,  "Hardware  Implementation  of 
Hierarchical  Volume  Subdivision-based  Elastic  Registration"  in  28th  Annual 
International  Conference  of  the  IEEE:  Engineering  in  Medicine  and  Biology 
Society,  2006,  pp.  1425-1428. 

■  O.  Dandekar,  W.  Plishker,  S.  Bhattacharyya,  and  R.  Shekahr,  "Multiobjective 
Optimization  of  FPGA-Based  Medical  Image  Registration"  IEEE  Symposium  on 
Field-Programmable  Custom  Computing  Machines,  Under  Review,  2008. 


32 


■  R.  Shekhar,  O.  Dandekar,  S.  Kavic,  I.  George,  R.  Mezrich,  and  A.  Park, 
"Development  of  continuous  CT-guided  minimally  invasive  surgery," 

Multimedia  Meets  Virtual  Reality  (MMVR),  2007. 

■  R.  Shekhar,  O.  Dandekar,  S.  Kavic,  I.  George,  R.  Mezrich,  and  A.  Park, 
"Development  of  continuous  CT-guided  minimally  invasive  surgery,"  Proc  SPIE, 
Medical  Imaging  2007. 

■  Seagull  FJ,  Xiao  Y,  &  Plasters  C.  Information  Accuracy  and  Sampling  Effort:  A 
Field  Study  of  Surgical  Scheduling  Coordination.  IEEE  Transactions  on 
Systems,  Man,  and  Cybernetics,  Part  A:Systems  and  Humans.  24(6),  764-771. 
2004 

■  Xiao  Y,  Dexter  F,  Hu  FP,  Dutton  R.  Usage  of  Distributed  Displays  of  Operating 
Room  Video  when  Real-Time  Occupancy  Status  was  Available  .  Anesthesia  and 
Analgesia  2008;  106(2):554-560.  2008 

■  Xiao  Y,  Hu  P,  Hu  H,  Ho  D,  Dexter  F,  Mackenzie  CF,  Seagull  FJ,  Dutton  D.  An 
algorithm  for  processing  vital  sign  monitoring  data  to  remotely  identify  operating 
room  occupancy  in  real-time.  Anesthesia  &  Analgesia, (101)3:823-829 . 2005 

■  Xiao  Y,  Hu  P,  Moss  J,  de  Winter  J,  Venekamp  D,  Mackenzie  CF,  Seagull  FJ, 
Perkins  S.  Opportunities  and  challenges  in  improving  surgical  work  flow. 
Cognition,  Technology  &  Work,  accepted.  2007 

■  Xiao  Y,  Schimpff  S,  Mackenzie  CF,  Merrell  R,  Entin  E,  Voigt  R,  Jarrell  B. 

Video  Technology  to  Advance  Safety  in  the  Operating  Room  and  Perioperative 
Environment.  Surgical  Innovation.  14(1):  52-61.  2007 

■  Xiao  Y,  Schimpff  S,  Mackenzie  CF,  Merrell  R,  Entin  E,  Voigt  R,  Jarrell  B. 

Video  Technology  to  Advance  Safety  in  the  Operating  Room  and  Perioperative 
Environment.  Surgical  Innovation.  14(1):  52-61.  2007 

■  Xiao  Y,  Strader  M,  Hu  P,  Wasei  M,  Wieringa  P.  Visualization  Techniques  for 
Collaborative  Trajectory  Management .  ACM  Conference  on  Human  Factors  in 
Computing  Systems,  pp.1547  -  1552.  2006 

■  Xiao  Y,  Wasei  M,  Hu  P,  Wieringa  P,  Dexter  F.  Dynamic  Management  in 
Perioperative  Processes:  A  Modeling  and  Visualization  Paradigm.  12th  IFAC 
Symposium  on  Information  Control  Problems  in  Manufacturing.  (3)647-52.  2006 

More  Recent  Publications 

ABSTRACT:  High-Speech  Reconstruction  of  Low-Dose  CT  Using  Iterative 
Techniques  for  Image-Guided  Interventions.  Doctoral  Dissertation,  University  of 
Maryland  2008.  Venkatesh  Bantwal  Bhat 

ABSTRACT:  High-Speech  Reconstruction  of  Low-Dose  CT  Using  GP-GPU. 
Venkatesh  Bhat  and  Raj  Shekhar 

ABSTRACT:  Live  Augmented  Reality  for  Laparoscopic  Surgery  Using  a  Novel 
Imaging  Method  -  Initial  Results  from  a  Porcine  Animal  Model.  R  Shekhar  PhD.  CD 
Godinez,  M.D.,  S  Kavic,  M.D.,  E  Hart,  M.D.,  Ivan  George,  AE  Park,  M.D. 


33 


Nagy  P,  Ramon  Konewko,  Max  Warnock,  Wendy  Bernstein,  Jacob  Seagull,  Yan 
Xiao,  Ivan  George,  and  Adrian  Park.  “Novel,  Web-Based,  Information-Exploration 
Approach  for  Improving  Operating  Room  Logistics  and  System  Processes”,  Surg 
Innov  2008  15:7-16. 

Ordonez  P,  P.  Kodeswaran,  V.  Korolev,  W.  Li,  O.  Walavalkar,  B. 

Elgamil,  A.  Joshi,  T.  Finin,  Y.  Yesha,  I. George.  “A  Ubiquitous 
Context-Aware  Environment  for  Surgical  Training”.  The  First  International 
Workshop  on  Mobile  and  Ubiquitous  Context  Aware  Systems  and  Applications 
(MUBICA  2007),  August  2007. 

Seagull  FJ,  Moses  GR,  Park  AE.  “Integration  of  Virtual  Reality  and  Conventional 
Skills  Trainers:  A  Mixed  Resource  Model.”  In  Westwood  J.D.,  Haluck  R.S., 
Hoffman  H.M.,  Mogel  G.T.,  Phillips  R.,  Robb  R.A.  and  Vosburgh  K.G.  (Eds). 

Studies  In  Health  Technology  and  Informatics  Volume  132.  Medicine  Meets  Virtual 
Reality  16  -  parallel,  combinatorial,  convergent:  NextMed  by  Design,  pp.  446-50. 
2008. 

Moses  GR  and  Park  AE.  “Ergonomic  risk  associated  with  assisting  in  minimally 
invasive  surgery.  ”  accepted  in  Westwood  J.D.,  Haluck  R.S.,  Hoffman  H.M.,  Mogel 
G.T.,  Phillips  R.,  Robb  R.A.  and  Vosburgh  K.G.  (Eds).  Studies  In  Health  Technology 
and  Informatics  Volume  132.  Medicine  Meets  Virtual  Reality  16,  2008. 

Dexter  F,  Xiao  Y,  Dow  AJ,  Strader  MM,  Ho  D,  Wachtel  RE.  “Coordination  of 
Appointments  for  Anesthesia  Care  Outside  of  Operating  Rooms  Using  an  Enterprise 
Wide  Scheduling  System”.  Anesthesia  and  Analgesia.  105:1701-1710.  2007 

Xiao  Y,  Schimpff  S,  Mackenzie  CF,  Merrell  R,  Entin  E,  Voigt  R,  Jarrell  B.  “Video 
Technology  to  Advance  Safety  in  the  Operating  Room  and  Perioperative 
Environment”.  Surgical  Innovation.  14(1):  52-61.  2007 

Xiao  Y,  Dexter  F,  Hu  FP,  Dutton  R.  “Usage  of  Distributed  Displays  of  Operating 
Room  Video  when  Real-Time  Occupancy  Status  was  Available”  .  Anesthesia  and 
Analgesia  2008;  106(2):554-560.  2008 

Kim  Y-J,  Xiao  Y,  Hu  P,  Dutton  RP.  “Staff  Acceptance  of  Video  Monitoring  for 
Coordination:  A  Video  System  to  Support  Perioperative  Situation  Awareness”. 
Journal  of  Clinical  Nursing  (accepted).  2007 

Dandekar  O.,  K.  Siddiqui,  V.  Walimbe,  and  R.  Shekhar,  "Image  registration  accuracy 
with  low-dose  CT:  how  low  can  we  go?,"  in  3rd  IEEE  International  Symposium  on 
Biomedical  Imaging:  Nano  to  Macro,  2006,  pp.  502-505. 

Dandekar  O.,  C.  Castro-Pareja,  and  R.  Shekhar,  "FPGA-based  real-time  3D  image 
preprocessing  for  image-guided  medical  interventions,"  Journal  of  Real-Time  Image 
Processing,  vol.  1(4),  pp.  285-301,  2007. 


34 


Shetye  A.S.  and  R.  Shekhar,  "A  statistical  approach  to  high  quality  CT  reconstruction 
at  low  radiation  doses  for  real-time  guidance  and  navigation,"  Proc.  SPIE  Med. 
Imaging,  2007. 

Dandekar  O.,  V.  Walimbe,  and  R.  Shekhar,  "Hardware  Implementation  of 
Hierarchical  Volume  Subdivision-based  Elastic  Registration"  in  28th  Annual 
International  Conference  of  the  IEEE:  Engineering  in  Medicine  and  Biology  Society, 
2006,  pp.  1425-1428. 

Dandekar  O.  and  R.  Shekhar,  "FPGA-accelerated  Defonnable  Registration  for 
Improved  Target-delineation  During  CT-guided  Interventions,"  IEEE  Transactions  on 
Biomedical  Circuits  and  Systems,  vol.  1(2),  pp.  1 16-127,  2007. 

Dandekar  O.,  W.  Plishker,  S.  Bhattacharyya,  and  R.  Shekahr,  "Multiobjective 
Optimization  of  FPGA -Based  Medical  Image  Registration"  IEEE  Symposium  on 
Field-Programmable  Custom  Computing  Machines,  Under  Review,  2008. 

Shekhar  R.,  O.  Dandekar,  S.  Kavic,  I.  George,  R.  Mezrich,  and  A.  Park, 

"Development  of  continuous  CT-guided  minimally  invasive  surgery,"  Multimedia 
Meets  Virtual  Reality  (MMVR),  2007. 

Shekhar  R.,  O.  Dandekar,  S.  Kavic,  I.  George,  R.  Mezrich,  and  A.  Park, 

"Development  of  continuous  CT-guided  minimally  invasive  surgery,"  Proc  SPIE, 
Medical  Imaging  2007. 

Lee  G.  and  Adrian  E.  Park.  “Development  of  a  More  Robust  Tool  for  Postural 
Stability  Analysis  of  Laparoscopic  Surgeons”.  Surg  Endosc  (2008)  22:1087-1092 

Lee  G.,  T.  Lee,  D.  Dexter,  R.  Klein,  and  A  Park.  “Methodological  Infrastructure  in 
Surgical  Ergonomics:  A  Review  of  Tasks,  Models,  and  Measurement  Systems”. 
Surgical  Innovation,  Volume  14  Number  3,  September  2007  153-167 

Lee  G.,  S.  M.  Kavic,  I.  M.  George  and  AE.  Park.  “Postural  Instability  Does  Not 
Necessarily  Correlate  to  Poor  Performance:  Case  in  Point”.  Received:  7  August 
2006/Accepted:  22  September  2006/Online  publication:  8  February  2007,  Surg 
Endosc (2007)  2 1 :  471-474 

Lee  G.,  T.  Lee,  D.  Dexter,  C.  Godinez,  N.  Meenaghan,  R.Catania  and  AE  Park. 
“Ergonomic  Risk  Associated  with  Assisting  in  Minimally  Invasive  Surgery”. 

Accepted  August  2008,  Surgical  Endoscopy 


35 


APPENDICES 


A.  Computer  Input  Devices:  Neutral  Party  or  Source  of  Significant  Error 
in  Manual  Lesion  Segmentation?  In  Journal  of  Digital  Imaging,  James  Y. 
Chen,  F.  Jacob  Seagull,  Paul  Nagy,  Paras  Lakhani,  Elias  R.  Melhem,  Eliot 
L.  Siegel,  and  Nabile  M.  Safdar. 


36 


Journal  of  Dioilal  Imaging 


Computer  Input  Devices:  Neutral  Party  or  Source  of  Significant  Error 

in  Manual  Lesion  Segmentation? 

James  Y.  Chen, 1-2-3  F.  Jacob  Seagull,1 2  Paul  Nagy,2  Paras  Lakhani,1-2  Elias  R.  Melhem,1  Eliot  L.  Siegel,2-4 

and  Nabile  M.  Safdar2 


Lesion  segmentation  involves  outlining  the  contour  of  an 
abnormality  on  an  image  to  distinguish  boundaries 
between  normal  and  abnormal  tissue  and  is  essential  to 
track  malignant  and  benign  disease  in  medical  imaging 
for  clinical,  research,  and  treatment  purposes.  A  laser 
optical  mouse  and  a  graphics  tablet  were  used  by 
radiologists  to  segment  12  simulated  reference  lesions 
per  subject  in  two  groups  (one  group  comprised  three 
lesion  morphologies  in  two  sizes,  one  for  each  input 
device  for  each  device  two  sets  of  six,  composed  of 
three  morphologies  in  two  sizes  each).  Time  for 
segmentation  was  recorded.  Subjects  completed  an 
opinion  survey  following  segmentation.  Error  in  contour 
segmentation  was  calculated  using  root  mean  square 
error.  Error  in  area  of  segmentation  was  calculated 
compared  to  the  reference  lesion.  11  radiologists  seg¬ 
mented  a  total  of  132  simulated  lesions.  Overall  error  in 
contour  segmentation  was  less  with  the  graphics  tablet 
than  with  the  mouse  LP<0.0001).  Error  in  area  of 
segmentation  was  not  significantly  different  between 
the  tablet  and  the  mouse  LP=0.62).  Time  for  segmen¬ 
tation  was  less  with  the  tablet  than  the  mouse  (P  = 
0.011).  All  subjects  preferred  the  graphics  tablet  for 
future  segmentation  (P=  0.011)  and  felt  subjectively 
that  the  tablet  was  faster,  easier,  and  more  accurate 
(P  =  0.0005).  For  purposes  in  which  accuracy  in  contour 
of  lesion  segmentation  is  of  the  greater  importance,  the 
graphics  tablet  is  superior  to  the  mouse  in  accuracy  with 
a  small  speed  benefit.  For  purposes  in  which  accuracy  of 
area  of  lesion  segmentation  is  of  greater  importance,  the 
graphics  tablet  and  mouse  are  equally  accurate. 

KEY  WORDS:  Image  segmentation,  user-computer 
interface,  computer  assisted  detection,  computer 
hardware,  data  collection,  human  computer  interaction, 
evaluation  research,  segmentation 


INTRODUCTION 

Lesion  segmentation  involves  outlining  the 
contour  of  an  abnormality  on  an  image  to 
distinguish  boundaries  between  normal  and  abnor¬ 


mal  tissue.  Segmentation  of  lesion  volume  (or  area 
on  2D  images  slices)  is  essential  to  track  malignant 
and  benign  disease  in  medical  imaging  for  clinical, 
research,  and  treatment  purposes. 

Clinically,  large  trials  such  as  the  ACRIN 
National  Lung  Screening  Trial  as  well  as  individual 
clinical  cases  currently  rely  on  accurate  and 
repeatable  methods  of  lesion  measurement  and 
segmentation. 1-5  Automated  tumor  segmentation 
for  radiation  therapy  can  improve  treatment  plan¬ 
ning  accuracy6  and  highly  localized  radiotherapies, 
such  as  proton  beam,7 *  require  accurate  pre¬ 
treatment  targeting.3’9 

For  research,  segmentation  can  be  used  for 
objective  comparison  of  new  imaging  sequences 
and  modalities  or  for  the  creation  of  automated 
segmentation  tools. 10-1 3  The  validation  of  auto¬ 
mated  segmentation  tools  commonly  relies  on 
testing  for  lesion  contour  and  size  against  manual 
segmentation. 14-1 9  Manual  lesion  segmentation, 
however,  may  vary  in  accuracy,  depending  on  the 
input  method  used  for  measurement.  To  our 
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knowledge,  nearly  all  clinical  and  research  users 
use  the  mouse  as  the  input  device  for  manual 
lesion  segmentation. 

The  QWERTY  keyboard  and  mouse  are  the  de 
facto  standard  configuration  in  computer  input 
devices.  Although  these  work  well  for  standard 
user  interface  interactions,  many  graphic  designers 
have  chosen  to  replace  or  augment  these  devices 
with  graphics  tablets  that  more  closely  mimic 
conventional  pen  and  paper.  Applications  that 
require  manual  lesion  segmentation  bear  similar¬ 
ities  to  the  tasks  of  graphic  designers,  chiefly  a 
need  for  finer  motor  control.  Most  people  perform¬ 
ing  segmentation,  however,  still  rely  mainly  on  the 
mouse  interface.  The  mouse  interface  can  be 
variably  accurate  and  used  with  either  finer  finger 
and  wrist  movements  or  larger  arm  and  wrist 
movements;  whereas,  the  accuracy  of  pen  strokes 
is  typically  limited  to  fine  movements  such  as 
wrist  flexion. 

To  our  knowledge,  at  the  time  of  the  writing  of 
this  manuscript,  there  has  been  a  single  published 
medical  study  comparing  the  accuracy  of  the 
computer  mouse  with  alternative  input  devices 

including  the  graphics  tablet  and  touch-sensitive 
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screen. 

We  hypothesize  that  the  pen-and-tablet  interface 
should  be  empirically  and  subjectively  at  least  as 
accurate,  easy,  and  fast  as  an  optical  mouse  for 
lesion  segmentation  in  regards  to  both  lesion 
contour  and  size. 


MATERIALS  AND  METHODS 

This  is  an  IRB-approved  prospective  study  com¬ 
paring  two  different  computer  input  devices  for 
manual  segmentation  of  simulated  lesions,  including 
a  post-experiment  survey  of  all  participants. 

Simulated  reference  lesions  were  created  and  then 
combined  into  one  set  of  two  groups  of  six  images. 
One  group  comprised  of  three  lesion  morphologies 
in  two  sizes,  one  for  each  input  device  (Fig.  1). 
Lesion  contours  were  created  to  simulate  clinical 
lesions,  including  ovoid,  lobulated,  and  spiculated 
forms.  To  account  for  differences  in  lesion  size/ 
zooming,  three  lesion  shapes  were  resized  50%  in 
each  dimension  using  bicubic  interpolation  to  create 
a  second  group  of  three  smaller  size  lesions.  Each 
image  contained  only  a  single  lesion  with  high- 
contrast,  black-on-white  backgrounds,  and  hard 
edges  to  minimize  the  cognitive  task  and  time 
required  to  identify  the  lesion  and  its  borders. 

A  wireless,  800  dpi,  laser  mouse  (Logitech 
International  S.A.;  Switzerland)  using  the  default 
software  driver  (Microsoft  Corporation,  version 
5.1.2600.5512)  was  set  to  default  movement 
parameters  including  variable  gain/acceleration. 
The  batteries  were  used  for  less  than  1  week 
during  the  experiment  to  minimize  performance 
degradation  from  battery  exhaustion,  and  the  same 
non-reflective  mouse  surface  was  used  for  all 
subjects.  The  graphics  tablet  (Wacom  Corporation; 
Japan)  was  also  set  to  default  parameters  within 
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Fig  1 .  Three  lesion  shapes  (left)  were  used  as  templates  to  create  high-contrast,  hard-edged  images  in  two  sizes  each.  Study  participants 
segmented  one  set  of  six  images  with  the  mouse  and  the  other  set  of  six  images  with  the  graphics  tablet. 
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the  driver  software  (Wacom  Corporation,  version 
6.00-5)  and  matched  the  aspect  ratio  of  the  view¬ 
ing  screen.  Subjects  were  encouraged  to  position 
both  devices  and  display  for  their  comfort. 

Radiologists  performed  manual  lesion  segmen¬ 
tation  with  each  input  device  for  each  group  of  six 
images  (12  lesion  segmentations  per  subject)  using 
commercially  available  image  editing  software 
(Adobe  Systems,  USA).  Simulated  lesions  and 
segmentation  device  order  were  randomized  for 
each  subject.  Participants  were  allowed  a  single 
image  on  which  to  practice  segmentation  up  to  two 
times  with  each  input  device  prior  to  segmenting 
each  experimental  set.  Segmentation  time  was 
recorded  manually  for  each  individual  lesion  by  a 
single  observer  throughout  the  study. 

To  evaluate  error  in  contour  of  segmented 
lesions,  undersegmented  and  oversegmented  areas 
were  combined  to  assess  the  mis-sampled  area. 
Undersegmented  areas  were  defined  as  areas  of 
reference  lesion  that  was  not  included  in  manual 
segmentation  (false  exclusion).  Oversegmented 
areas  were  defined  as  areas  of  segmented  lesion 
that  did  not  correlate  to  the  reference  lesion  (false 
inclusion).  The  segmented  lesions  were  subtracted 
from  the  reference  lesion  to  obtain  areas  of 
undersegmentation;  the  reference  lesion  was  then 
subtracted  from  the  segmented  lesion  to  obtain  the 
area  of  oversegmentation.  The  root  mean  square 
error  was  then  calculated  from  the  undersegmented 
and  oversegmented  areas.  Statistical  differences 
were  compared  with  a  Wilcoxon  test. 

To  evaluate  error  in  area  of  segmented  lesions, 
total  area  of  segmentation  was  recorded  for  each 
lesion  and  the  difference  in  area  from  the  reference 
lesion  was  calculated.  These  differences  were 
compared  with  a  Wilcoxon  test. 

After  segmentation  of  both  lesion  sets,  each 
individual  completed  a  force-choice  survey  between 


the  graphics  tablet  and  mouse  for  perceived  ease, 
speed,  and  accuracy  of  segmentation;  and  device 
preference  for  segmenting  lesions  in  the  future. 
Statistical  differences  in  preferences  were  compared 
with  a  binomial  test.  Subject  demographics  were 
recorded  during  the  survey  including  number  of 
years  of  experience  with  a  mouse  or  tablet. 

RESULTS 

Eleven  radiologists  (nine  male,  two  female)  partici¬ 
pated  in  the  study,  segmenting  a  total  of  132  lesions. 

Average  lesion  area  was  16,041  pixels  with 
large  lesions  averaging  22,203  pixels,  and  small 
lesions  averaging  5,508  pixels.  Ovoid  lesions 
averaged  12,562  pixels,  lobulated  lesions  18,569 
pixels,  and  spiculated  lesions  10,437  pixels. 

Contour  segmentation  with  the  tablet  was  more 
accurate  than  with  a  mouse  (Table  1)  with  average 
RMS  error  of  690  (standard  deviation  of  530, 
overall  RMS  of  4.3%  of  average  lesion  area) 
versus  992  (standard  deviation  of  1,033,  overall 
RMS  of  6.2%  of  average  lesion  area,  an  increase 
of  44%  in  error;  P<0.0001).  For  the  large  lesions, 
the  tablet  demonstrated  significantly  less  error 
(RMS  error  of  1,003,  standard  deviation  of  617, 
overall  RMS  of  4.5%  of  large  lesion  area) 
compared  to  the  mouse  (RMS  error  of  1,489, 
standard  deviation  of  1,288,  overall  RMS  of  6.7% 
of  average  large  lesion  area,  corresponding  to  a 
48%  increase  in  error;  JD<0.0001).  For  small 
lesions,  the  tablet  also  demonstrated  significantly 
less  error  (RMS  error  of  377,  standard  deviation  of 
246,  overall  RMS  of  6.8%  of  small  lesion  area) 
compared  to  the  mouse  (RMS  error  of  496, 
standard  deviation  of  390,  overall  RMS  of  9.0% 
of  small  lesion  area,  corresponding  to  a  32% 
increase  in  error;  T’=0.0121).  Of  the  11  partic- 


Table  1 .  Root  Mean  Square  Error  in  Contour  Error  as  a  Percent  of  Reference  Lesion  Area 


Contour  segmentation  error  between  graphics  tablet  and  mouse 


Lesion  set 

Tablet 

Mouse 

Increase  from  tablet 

P 

All  lesions 

4.3% 

6.2% 

44% 

<0.0001 

Large  lesions 

4.5% 

6.7% 

48% 

<0.0001 

Small  lesions 

6.8% 

9.0% 

32% 

0.0121 

Ovoid  lesions 

2.9% 

4.6% 

59% 

0.0008 

Lobulated  lesions 

3.8% 

5.1% 

33% 

0.0127 

Spiculated  lesions 

9.3% 

14% 

46% 

0.0156 

There  is  significantly  greater  contour  error  with  a  mouse  across  all  lesions  and  lesion  subgroups 
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ipants,  ten  (91%)  were  more  accurate  with  the 
tablet  compared  to  one  (9%)  with  the  mouse. 

When  lesions  were  subgrouped  into  ovoid,  lobu¬ 
lated,  and  spiculated  morphologies,  the  tablet  again 
demonstrated  less  contour  error  than  the  mouse, 
except  with  the  lobulated  lesions.  For  lobulated 
lesions,  the  tablet  demonstrated  significantly  less 
error  (P=0.0127).  Tablet  error  (RMS  of  714,  stand¬ 
ard  deviation  of 498,  overall  RMS  3.8%  of  lobulated 
lesion  area)  compared  to  the  mouse  (RMS  of  950, 
standard  deviation  of  898,  overall  RMS  of  5.1%  of 
lobulated  lesion  area,  corresponding  to  33%  increase 
in  error  of  33%).  With  ovoid  lesions,  the  tablet 
demonstrated  significantly  less  error  (RMS  of  364, 
standard  deviation  of  294,  overall  RMS  2.9%  of 
ovoid  lesion  area)  compared  to  the  mouse  (RMS  of 
580,  standard  deviation  of 574,  overall  RMS  4.6%  of 
ovoid  lesion  area,  corresponding  to  a  59%  increase  in 
error;  P=0.0008).  With  spiculated  lesions,  the  tablet 
demonstrated  significantly  less  error  (RMS  of  975, 
standard  deviation  of 682,  overall  9.3%  of  spiculated 
lesion  area)  compared  to  the  mouse  (RMS  of  1 ,424, 
standard  deviation  of  1,431,  overall  13.6%  of 
spiculated  lesion  area  corresponding  to  a  46% 
increase  in  error;  P=().()  1 56). 

Error  in  pixel  area  of  segmentation  with  the 
tablet  and  mouse  was  not  significantly  different 
(Table  2).  Across  all  lesions,  the  tablet  demon¬ 
strated  a  mean  error  in  area  of  678  pixels  (4.9%  of 
lesion  area  and  standard  deviation  of  920  pixels) 
compared  to  617  pixels  (4.5%  of  lesion  area  and  a 
standard  deviation  of  1,119  pixels)  for  the  mouse 
(P= 0.6194).  For  large  lesions,  the  tablet  demon¬ 
strated  a  mean  error  in  area  of  1,052  pixels  (4.7% 
of  lesion  area  and  standard  deviation  of  1,114 
pixels)  compared  to  954  pixels  for  the  mouse 
(4.3%  of  lesion  area  and  standard  deviation  of 
1,402  pixels;  F>=0.4658).  For  small  lesions,  the 


Table  2.  Pixel  Area  Error  as  a  Percent  of  Reference  Lesion  Pixel 
Area 


Pixel  area  segmentation  error  between  graphics  tablet  and  mouse 

Lesion  set 

Tablet 

Mouse 

P 

All  lesions 

4.9% 

4.5% 

0.619 

Large 

4.7% 

4.3% 

0.466 

Small 

5.8% 

5.8% 

0.844 

Ovoid 

2.9% 

-0.1% 

0.248 

Lobulated 

1.9% 

2.7% 

0.849 

Spiculated 

13% 

14% 

0.838 

There  is  no  significant  difference  in  pixel  area  error  between  the 
graphics  tablet  and  mouse 


tablet  demonstrated  a  mean  error  in  area  of  318 
pixels  (5.8%  of  lesion  area  and  standard  deviation 
of  458  pixels)  compared  to  322  pixels  for  the 
mouse  (5.8%  of  lesion  area  and  standard  deviation 
of  628  pixels;  P=0.8442).  For  ovoid  lesions,  the 
tablet  demonstrated  a  mean  error  in  area  of  364 
pixels  (2.9%  of  lesion  area  and  standard  deviation 
of  555  pixels),  compared  to  -17  pixels  for  the 
mouse  (-0.1%  of  lesion  area  and  standard  devia¬ 
tion  of  998  pixels;  P=0.2479).  For  lobulated 
lesions,  the  tablet  demonstrated  a  mean  error  in 
area  of  362  pixels  (1.9%  of  lesion  area  and 
standard  deviation  of  875  pixels)  compared  to 
499  pixels  for  the  mouse  (2.7%  of  lesion  area  and 
standard  deviation  of  593  pixels;  P=0.8486).  For 
spiculated  lesions,  the  tablet  demonstrated  a  mean 
error  in  area  of  1,309  pixels  (12.5%  of  lesion  area 
and  standard  deviation  of  958  pixels)  compared  to 
1,421  pixels  for  the  mouse  (13.6%  of  lesion  area 
and  standard  deviation  of  1,184  pixels;  />=0.8382). 
Of  the  11  participants,  seven  were  more  overall 
more  accurate  with  the  tablet  (64%)  than  with  the 
four  with  the  mouse  (36%). 

Overall  lesion  segmentation  time  was  statisti¬ 
cally  significantly  less  with  the  tablet,  averaging 
27  s  versus  29  s  with  the  mouse  (/>=0.031,  R2= 
0.96).  For  the  small  lesions,  the  tablet  (23  s)  was 
significantly  faster  than  the  mouse  (25  s;  P= 
0.011).  There  was  no  significant  difference 
between  the  tablet  and  mouse  when  other  subsets 
were  analyzed.  For  the  large  lesions,  there  was  no 
significant  difference  between  the  tablet  (32  s)  and 
mouse  (34  s;  P=0.24).  There  was  no  difference  in 
segmentation  time  for  ovoid  lesions  between  the 
tablet  (15  s)  and  mouse  (16  S;  P=0.41),  for 
lobulated  lesions  between  the  tablet  (24  s)  and 
mouse  (27  s;  P=  0.22),  or  for  spiculated  lesions 
(42  s)  versus  (45  s)  (P=0.13). 

All  radiologists  reported  more  experience  using 
a  mouse  than  a  tablet,  with  100%  having  >5  years 
of  mouse  experience.  Only  three  subjects  (27%) 
had  previously  used  a  graphics  tablet,  two  (18%) 
reported  up  to  1  year  of  experience,  and  one  (9%) 
reported  3-5  years  of  experience. 

Nine  (82%)  reported  the  perception  that  seg¬ 
mentation  with  tablet  was  faster,  while  one  (9%) 
perceived  the  mouse  to  be  faster,  and  one  (9%) 
abstained  from  the  question  (P=0.011). 

All  (100%)  participants  reported  segmentation 
with  a  mouse  to  be  more  difficult  as  well  as  less 
accurate.  All  (100%)  participants  also  reported  a 
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preference  for  using  a  graphic  tablet  in  the  future 
when  segmenting  lesions  (,P=0.0005). 

DISCUSSION 

A  significant  decrease  in  contour  error  was  seen 
when  manually  segmenting  lesions  with  a  graphics 
tablet  compared  with  the  same  task  perfonned  with 
a  mouse,  but  no  significant  difference  in  pixel  area 
error.  Despite  a  general  lack  of  experience  with  a 
graphics  tablet,  there  was  a  small,  but  statistically 
significant  decrease  in  time  to  segment  lesions 
between  the  graphics  tablet  and  mouse.  All 
operators  reported  the  graphics  tablet  to  be 
subjectively  more  accurate  and  easier,  and  all 
preferred  the  tablet  for  future  task  performance. 
This  suggests  that  with  the  same  amount  or 
decreased  time  expenditure,  lesions  can  be  seg¬ 
mented  more  accurately  in  contour  or  equally 
accurately  in  area  with  less  effort  than  with  a 
mouse,  a  finding  that  could  have  significant 
implications  for  research  or  clinical  work  involv¬ 
ing  large  numbers  of  lesions.  These  results  mirror 
findings  in  the  computing  literature  that  the 
tablet  input  device  is  more  accurate  for  precision 
movements.21’22 

While  the  contour  segmentation  error  as  a 
percentage  of  total  lesion  area  was  relatively  small, 
ranging  from  an  average  of  5-14%  for  the  mouse 
and  2.9-9%  for  the  tablet,  this  was  for  a  single 
image  in  2D  space.  Actual  pathology  occurs  in  3D 
space  and  accurate  assessment  of  lesion  volume 
would  require  segmenting  a  stack  of  2D  images 
and  adding  the  areas  to  calculate  3D  volume.  Even 
these  small  errors  accumulated  over  a  stack  of 
images  can  potentially  result  in  significant  cumu¬ 
lative  error.  In  clinical  trials  where  relatively  small 
differences  are  expected  of  treatment  efficacy, 
even  these  small  amounts  of  error  can  have 
potentially  large  effects  on  study  data  and  results. 
For  highly  targeted  radiotherapeutic  treatments, 
small  errors  in  targeting  can  result  in  under¬ 
treatment  of  a  lesion  or  damage  to  non-pathologic 
structures  adjacent  to  the  intended  target. 

When  error  in  the  total  area  of  segmentation  was 
compared,  there  was  no  statistically  significant 
difference  between  the  graphics  tablet  and  the 
mouse.  For  purposes  in  which  total  lesion  size  is 
the  subject  of  interest  over  lesion  contour,  such  as 
in  clinical  evaluation  of  lesion  size  for  treatment 


response  or  dosing  of  medication,  the  mouse  and 
tablet  can  be  considered  equivalent. 

The  perceived  increased  ease  of  lesion  segmen¬ 
tation  may  magnify  differences  between  the  mouse 
and  graphics  tablet  when  large  volumes  of  lesions 
are  segmented  within  and  across  patients.  One  set 
of  investigators  have  found  that  a  pen  tablet 
system  creates  less  overall  muscular  load  than  a 
mouse,2’’  while  another  found  improved  produc¬ 
tivity  in  general  cursor  control.24  Operator  fatigue 
and  frustration  with  the  input  method  during 
segmentation  can  lead  to  less  accurate  segmentation 
when  the  task  is  performed  repeatedly.  If  segmenta¬ 
tion  with  the  tablet  is  truly  easier,  the  graphics  tablet 
could  potentially  decrease  the  amount  of  and  effects 
of  fatigue.  This  could  result  in  creating  more  accurate 
datasets  with  large  numbers  of  segmented  lesions 
against  which  to  evaluate  automated  methods  of 
segmentation  and  new  imaging  sequences  or  modal¬ 
ities  as  they  are  developed. 

We  speculate  that  additional  experience  with  the 
tablet  beyond  that  held  by  the  participants  in  this 
study  might  increase  the  speed  of  lesion  segmen¬ 
tation  but  only  minimally  affect  accuracy — an  area 
that  remains  for  future  evaluation. 


LIMITATIONS 

Manual  timing  of  lesion  segmentation  time 
introduces  the  possibility  that  the  differences  in 
segmentation  time  were  at  least  partially,  if  not, 
wholly  related  to  variability  in  relying  on  a  human 
observer.  As  a  single  human  observer  performed 
timing  measurements  for  all  data  points,  variability 
was  limited  to  that  single  observer  instead  of 
across  multiple  observers.  Another  study  found 
that  the  tablet  is  faster  than  the  mouse  for  accurate 
clinical  contouring.-’’ 

The  study  design  was  not  conducive  to  a 
meaningful  analysis  of  intra-observer  variability, 
as  no  participant  perfonned  the  same  task  more 
than  once.  Any  analysis  of  intraobserver  variability 
would  therefore  be  confounded  by  additional 
factors,  such  as  size  and/or  shape  of  the  segmented 
lesion,  and  the  device  used.  This  design  was 
chosen  to  limit  the  total  time  required  by  the 
participants. 

Study  participants  were  not  allowed  to  correct 
their  lesion  segmentations  post  hoc.  In  clinical  use, 
radiologists  have  the  opportunity  to  correct  their 
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lesions  segmentations  regardless  of  the  input 
device  used,  but  the  frequency  and  total  amount 
of  correction  are  not  known.  An  input  device 
which  introduces  greater  error  would  also  increase 
the  time  and  labor  burden  of  correcting  the 
segmentation  potentially  decreasing  the  incentive 
for  error  correction.  This  could  lead  to  the  creation 
of  segmented  data  sets  with  greater  total  error. 

Our  generalizability  to  true  clinical  images  and 
lesion  segmentation  is  currently  unknown,  but  is  a 
potential  area  for  future  investigation.  Lesions  seen 
in  clinical  imaging  studies  can  be  less  conspicuous 
and  may  have  less  well-defined  margins.  This  can 
complicate  the  cognitive  task  of  defining  a  lesion’s 
presence  and  borders,  but  we  expect  that  this 
would  affect  the  use  of  both  the  graphics  tablet  and 
mouse  equally. 

Two  of  our  subjects  expressed  a  dislike  for  the 
specific  mouse  and  mouse-surface  used  in  this  study. 
The  mouse  and  surface  chosen,  however,  were 
popularly  available,  commercial  products  with  con¬ 
ventional  designs,  which  are  expected  to  be  similar  to 
the  default  devices  provided  for  clinical  use.  It  is 
unknown  whether  this  is  related  to  the  specific 
devices  or  related  to  a  preference  of  different  driver 
settings.  In  this  study,  default  settings  were  chosen, 
as  these  were  felt  to  be  the  most  commonly  used  in 
clinical  workstations.  The  effect  of  the  specific 
mouse,  mouse  surface,  and  driver  settings  may  be  a 
topic  for  future  investigation. 


CONCLUSION 

The  choice  of  input  device  for  manual  lesion 
segmentation  was  found  to  significantly  affect 
accuracy  of  segmentation  of  lesion  contour,  but  not 
total  lesion  size.  These  findings  suggest  that  for 
specific  purposes  in  which  lesion  contour  is  of 
greater  importance,  such  as  in  comparing  different 
imaging  sequences  and  modalities  or  for  targeting 
highly  selective  radiotherapy,  there  is  significantly 
greater  error  with  a  mouse  than  with  a  graphics  tablet 
at  a  slight  speed  penalty.  In  purposes  in  which  total 
lesion  size  is  of  greater  importance,  the  mouse  and 
tablet  are  equally  accurate  for  lesion  segmentation. 
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^ “  Radiology  departments  today  are  faced  with  many  challenges  to  im¬ 

prove  operational  efficiency,  performance,  and  quality.  Many  organi¬ 
zations  rely  on  antiquated,  paper-based  methods  to  review  their  his¬ 
torical  performance  and  understand  their  operations.  With  increased 
workloads,  geographically  dispersed  image  acquisition  and  reading 
sites,  and  rapidly  changing  technologies,  this  approach  is  increasingly 
untenable.  A  Web-based  dashboard  was  constructed  to  automate  the 
extraction,  processing,  and  display  of  indicators  and  thereby  provide 
useful  and  current  data  for  twice-monthly  departmental  operational 
meetings.  The  feasibility  of  extracting  specific  metrics  from  clinical 
information  systems  was  evaluated  as  part  of  a  longer-term  effort  to 
build  a  radiology  business  intelligence  architecture.  Operational  data 
were  extracted  from  clinical  information  systems  and  stored  in  a  cen¬ 
tralized  data  warehouse.  Higher-level  analytics  were  performed  on 
the  centralized  data,  a  process  that  generated  indicators  in  a  dynamic 
Web-based  graphical  environment  that  proved  valuable  in  discussion 
and  root  cause  analysis.  Results  aggregated  over  a  24-month  period 
since  implementation  suggest  that  this  operational  business  intelli¬ 
gence  reporting  system  has  provided  significant  data  for  driving  more 
effective  management  decisions  to  improve  productivity,  performance, 
and  quality  of  service  in  the  department. 

®RSNA,  2009  •  radiographics.rsna.org 


Abbreviations:  DICOM  =  Digital  Imaging  and  Communications  in  Medicine,  FTP  =  File  Transfer  Protocol,  HL7  =  Health  Level  7,  MPPS  = 
Modality  Performed  Procedure  Step,  ODBC  =  Open  Database  Connection,  PACS  =  picture  archiving  and  communication  system,  RIS  =  radiology 
information  system 
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Introduction 

The  field  of  business  intelligence,  also  known  as 
business  analytics,  has  demonstrated  significant 
improvements  in  industries  outside  of  healthcare 
(1).  Business  intelligence  tools  enable  manage¬ 
ment  to  make  more  relevant  decisions  with 
greater  frequency,  facilitating  smaller  course  cor¬ 
rections  to  keep  the  business  improving  and  min¬ 
imizing  oversights.  The  two  principal  benefits  of  a 
business  intelligence  solution  that  can  affect  the 
culture  of  an  organization  are  transparency  and 
fact-based  decision  making  (2).  Supreme  Court 
justice  Louis  Brandeis  noted  that  “sunlight  is  said 
to  be  the  best  of  disinfectants”  (3)  as  a  remedy 
for  social  diseases.  Transparency  can  be  used  as  a 
powerful  tool  to  provide  accountability  and  own¬ 
ership  of  performance  improvements. 

A  focus  on  fact-based  decision  making  changes 
traditional  response  models  based  on  emotions 
and  anecdotal  perceptions  of  service.  Sharing  data 
and  analysis  is  seen  as  a  powerful  way  to  influence 
other  key  stakeholders  into  aligning  to  a  common 
vision  of  a  problem.  Creating  an  open  culture  that 
is  willing  to  take  an  unwavering  look  at  the  “brutal 
facts”  of  an  organization  is  seen  as  a  key  charac¬ 
teristic  of  a  competitive  company  (4) .  Clinical 
information  systems  used  in  radiology  today  house 
a  treasure  trove  of  operations  data  that  is  not  cur¬ 
rently  being  used  by  clinical  management  to  create 
a  culture  of  quality  and  enable  a  more  engaged 
and  enlightened  management  environment. 

The  need  for  management  reporting  in  radi¬ 
ology  dates  back  to  the  1970s  as  one  of  the  core 
components  identified  in  early  radiology  infor¬ 
mation  system  (RIS)  implementations.  In  1979, 
Arenson  and  London  (5)  described  the  need  for  a 
computer  system  to  provide  operations  manage¬ 
ment  to  perform  time-flow  analysis  to  improve 
efficiency.  A  RIS  provided  tabulated  reports  to  be 
printed  out.  These  would  evolve  into  reports  that 
could  be  e-mailed  for  monthly  reporting  processes. 

In  1994,  Crabbe  et  al  (6)  described  a  manual 
extraction  methodology  for  obtaining  data  from  a 
RIS  that  provided  the  means  to  perform  higher- 
level  analysis  on  data  as  well  as  the  ability  to  dis¬ 
tribute  graphical  analysis  in  the  form  of  Excel  files 
(Microsoft,  Redmond,  Wash)  from  a  central  file 
server.  In  2000,  Seltzer  et  al  (7)  used  a  Web  site  as 
a  digital  dashboard  to  communicate  operational 
parameters.  In  2003,  our  group  used  automated 
extraction  methodologies  to  dynamically  build 


Web  reporting  capabilities  for  performance  and 
utilization  management  off  a  single  picture  ar¬ 
chiving  and  communication  system  (PACS)  (8). 

In  this  article,  we  describe  a  Web-based  dash¬ 
board  that  was  constructed  to  automate  the 
extraction,  processing,  and  display  of  indicators 
and  thereby  provide  useful  and  current  data  for 
departmental  operational  meetings.  In  addition, 
we  evaluate  the  feasibility  of  extracting  specific 
metrics  from  clinical  information  systems  as  part 
of  a  longer-term  effort  to  build  a  radiology  busi¬ 
ness  intelligence  architecture.  Specific  topics  dis¬ 
cussed  are  problems  with  a  paper-based  reporting 
method,  data  extraction  architecture,  information 
visualization,  and  enabling  knowledge  discovery. 


Problems  with  a 
Paper-based  Reporting  Method 

To  understand  the  need  for  an  online  dash¬ 
board,  we  first  identified  the  department’s  many 
frustrations  with  the  traditional  paper-based 
processes  of  analysis,  reporting,  and  quality 
management.  Many  significant  challenges  limit 
the  effectiveness  of  the  standard  methodology  of 
monthly  paper-based  reporting.  The  time  and 
effort  required  to  build  reports  creates  a  capture 
latency  that  diminishes  the  value  of  the  informa¬ 
tion  as  well  as  the  time  the  organization  then  has 
to  take  action  to  identify  and  implement  reme¬ 
dies  (2).  Paper-based  reporting  can  answer  only 
a  relatively  small  list  of  questions  that  must  be 
identified  before  the  monthly  (or  other  periodic) 
meeting  that  routinely  addresses  such  questions. 
This  finite  list  limits  the  focus  in  such  meetings 
to  the  data  at  hand  and  prevents  any  attempt  to 
follow  outside  data,  identify  disturbing  trends 
that  have  not  yet  become  true  issues,  or  delve 
deeper  into  operations.  When  questions  are 
asked  that  cannot  be  answered,  they  are  taken 
“off-line”  and  answered  at  the  next  meeting  if 
remembered  at  all.  This  cumbersome  and  some¬ 
times  quite  subjective  process  slows  the  ability 
of  an  organization  to  respond  quickly  to  events 
or  to  plan  effectively  for  change. 

Paper-based  reporting  lacks  the  granularity  to 
drill  down  on  a  metric  and  view  the  original  data. 
The  ability  to  view  the  actual  data  from  which 
statistical  analyses  are  derived  offers  a  higher 
level  of  confidence  in  understanding  data  and 
making  subsequent  decisions.  It  can  help  detect 
bias  such  as  skewed  distributions  or  data  integrity 
issues  that  might  affect  decisions  as  well  as  avoid 
anecdotal  mistrust  of  the  data. 
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Metrics  Selected  for  Display  on  the  Dashboard 

Metric  Label 

Group 

Description 

%  Outpatient 
Arrived 

Order  and  arrival 

Compliance  metric;  tracks  percentage  of  time  the  front  desk  enters  an 
arrival  time  for  outpatients 

Outpt.  %  Seen 
<15  min 

Order  and  arrival 

Performance  metric;  tracks  percentage  of  patients  taken  back  for  their 
examinations  within  1 5  min 

Avg  PI  CC  Time 
to  Arrival 

Order  and  arrival 

Performance  metric;  tracks  average  time  in  hours  for  placement  of  a 
PICC,  from  the  time  an  order  is  received  to  the  patient’s  arrival  at 
the  unit  for  PICC  placement 

Avg  STAT  TAT* 

Order  and  arrival 

Tracks  average  time  from  receipt  of  a  high-priority  order  to  completion 
of  image  acquisition 

%  Outpatient 
Begin 

Image  acquisition 

Compliance  metric;  tracks  percentage  of  time  the  technologists  enter  a 
procedure  begin  time 

%  Wait  >  1  h 

Image  acquisition 

Performance  metric;  tracks  percentage  of  patients  waiting  for  more  than 

1  h  to  be  seen 

QC  Issues 

Image  acquisition 

Tracks  number  of  quality  control  issues  submitted  by  radiologists  (13)f 

Image  Quality 

Image  acquisition 

Tracks  perceived  image  quality  as  determined  by  means  of  technologist 
peer  review 

Repeat  Rate 

Image  acquisition 

Tracks  self-reported  image  reject  and  retake  reasons 

Undictated 

>  1  mo 

Interpretation 

Billing  metric;  tracks  number  of  studies  performed  with  no  report  dic¬ 
tated  after  1  mo  by  interventional  radiologists 

Average  C-P  (h) 

Interpretation 

Performance  metric;  tracks  average  time  in  hours  from  completion  of  a 
study  to  completion  of  the  preliminary  report 

%  Peer  Reviewed 

Interpretation 

Compliance  metric;  tracks  percentage  of  reports  on  which  a  radiologist 
performed  peer  review 

Res  Review 

Submissions 

Interpretation 

Tracks  the  number  of  resident  reports  that  were  discrepant  with  the 
attending  physician’s  report 

Unsigned  >  2  wk 

Reporting 

Outlier  metric;  tracks  number  of  unsigned  reports  more  than  2  wk  old 

Average  P-F  (h) 

Reporting 

Performance  metric;  tracks  average  time  in  hours  from  preliminary 
report  of  a  study  to  the  time  the  report  is  finalized 

EPR  Ratio 

Reporting 

Tracks  number  of  formatting  errors  detected  in  the  final  report  sent 
from  the  speech  recognition  system 

Critical  Findings 
Delivery 

Reporting 

Tracks  number  of  critical  findings  documented 

Note. — EPR  =  errors  per  report,  PICC 

=  peripherally  inserted  central  catheter,  TAT  =  turnaround  time. 

*This  metric  excludes  the  emergency  department  and  shock  trauma  unit,  which  have  embedded  imaging  modalities. 
fNumber  in  parentheses  is  a  reference. 

In  addition,  paper-based  reports  are  usually 
constrained  to  those  “canned”  reports  available 
in  the  RIS.  These  predefined  reports  represent 
those  topics  that  RIS  vendors  expected  depart¬ 
ments  to  ask  for  at  the  time  of  software  develop¬ 
ment.  These  topics  and  the  elements  reported 
often  prove  inflexible  to  changing  business  and 
operational  needs.  Creating  custom  reports  is 
often  challenging,  preventing  departments  from 
answering  questions  and  fully  understanding 
their  operational  data. 

The  metrics  used  at  the  University  of  Maryland 
School  of  Medicine  before  implementation  of  our 
dashboard  efforts  were  similar  to  those  in  most 
academic  radiology  departments  (9-12).  We  evalu¬ 
ated  the  elements  of  the  paper-based  report  to 


determine  the  importance  and  difficulty  of  acquir¬ 
ing  these  metrics  from  clinical  information  sys¬ 
tems  in  an  automated  method.  The  Table  lists  the 
metrics  used  as  key  indicators  in  the  department 
on  the  dashboard.  Continuous  quality  improve¬ 
ment  requires  the  ability  to  change  the  metrics  on 
the  dashboard  as  well  as  the  targets  for  those 
metrics.  The  purpose  of  departmental  operational 
meetings  is  to  identify  problems  and  opportunities 
to  improve  performance.  As  the  department  im¬ 
proves,  metrics  can  be  retired  or  the  goals  can  be 
changed.  The  dashboard  must  be  able  to  incorpo¬ 
rate  the  moving-target  focus  of  the  meeting. 
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Figure  1.  Architecture  of  the 
data  warehouse  infrastructure. 
Clinical  information  systems 
were  interfaced  using  a  variety 
of  methods  to  extract  perfor¬ 
mance  and  quality  indicators. 
HTTP=  Hypertext  Transfer 
Protocol,  LDAP  =  Lightweight 
Directory  Access  Protocol. 
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Data  Extraction  Architecture 

One  of  the  challenges  to  providing  a  compre¬ 
hensive  view  of  operations  is  that  data  are  often 
distributed  among  several  different  information 
systems,  each  containing  only  discrete  pieces  of 
the  work  flow.  We  constructed  a  platform  to  col¬ 
lect  data  from  all  systems  to  build  a  comprehen¬ 
sive  view  of  operations  using  the  MySQL  open- 
source  database  as  the  central  data  repository 
(14). The  key  identifier  to  link  the  fragments  of 
data  together  was  the  unique  accession  number 
for  each  procedure. 

Data  extraction  tools  were  developed  in  the 
Python  programming  language  to  acquire  opera¬ 
tional  data  from  all  of  our  clinical  information 
systems,  using  standards  as  well  as  customized 
extraction  methodologies.  We  leveraged  two  pow¬ 
erful  open-source  tools,  the  DCM4CHE  Digital 
Imaging  and  Communications  in  Medicine 
(DICOM)  archive  (15)  and  the  MIRTH  Health 
Level  7  (HL7)  server  (16).  A  DICOM  repository 
was  created  using  DCM4CHE  to  collect  DICOM 
Modality  Performed  Procedure  Step  (MPPS) 
messages  automatically  routed  from  a  PACS  (6.2 
IMPAX;  Agfa,  Brussels,  Belgium).  HL7  messages 
of  orders  and  reports  were  routed  from  a  RIS 
(IDXRAD  9.0;  GE  Medical  Systems,  Fairfield, 
Conn)  to  the  MIRTH  HL7  report  repository. 


For  non-standard-based  reporting,  the  extrac¬ 
tion  methodology  had  to  be  tailored  to  the  type 
of  information  and  vendor  from  which  the  data 
were  derived.  Custom  reports  generated  from 
the  RIS  using  their  KBSQL  reporting  module 
stored  reports  in  comma-delimited  files.  An 
extraction  script  retrieved  those  files  at  regular 
intervals  via  File  Transfer  Protocol  (FTP),  parsed 
the  information,  and  uploaded  the  data  into  the 
MySQL  database.  The  PACS  was  queried  using  a 
metadata  extraction  Web  service  provided  by  the 
PACS  to  obtain  information  within  the  DICOM 
objects,  such  as  the  number  of  images  and  the 
performing  physician. 

A  speech  recognition  system  (RadWhere  2.0; 
Nuance  Healthcare  Solutions,  Burlington,  Mass) 
provided  data  via  an  Open  Database  Connection 
(ODBC)  with  an  account  restricted  to  only 
SELECT  Structured  Query  Language  state¬ 
ments  for  stability  and  security  purposes.  Quality 
tools  developed  within  the  department,  such  as 
quality  control  issue  tracking,  were  also  incorpo¬ 
rated  into  the  architecture  of  the  data  warehouse. 
Because  the  data  warehouse  contained  personal 
health  information,  clinical  security  precautions 
defined  by  the  Health  Insurance  Portability  and 
Accountability  Act  were  necessary,  and  appropri¬ 
ate  security  measures  were  enacted. 

The  interfaces  were  developed,  specified,  and 
tested  over  several  months.  For  each  interface, 
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the  commercial  vendors  were  consulted  and  the 
reporting  methods  were  specified.  No  develop¬ 
ment  was  requested  from  the  vendors.  The  easi¬ 
est  interfaces  to  implement  were  those  based 
on  open  standards,  such  as  HL7  and  DICOM. 
The  FTP  interface  was  straightforward,  as  pre¬ 
packaged  reports  were  retrieved  via  FTP.  The 
Web  service  interfaces  used  publicly  available 
application  programming  interfaces  that  were 
published  and  maintained  by  the  vendors.  The 
most  difficult  were  the  ODBC  interfaces,  which 
required  a  special  understanding  of  the  vendor’s 
database  structures  as  well  as  constrained  per¬ 
missions  and  testing  to  ensure  that  our  interface 
did  not  affect  the  performance  of  the  clinical 
system.  Figure  1  illustrates  the  architecture  and 
interfaces  employed. 

The  timing  of  data  extraction  depended  on 
how  frequently  the  data  were  being  reviewed 
and  how  actionable  the  information  was.  The 
goal  was  to  minimize  the  capture  latency  of  data 
acquisition.  The  main  tables  of  the  RIS  are  ex¬ 
tracted  every  evening  into  the  data  warehouse 
for  tracking  waiting  times,  turnaround  times, 
and  stat  order  times.  Order  and  report  informa¬ 
tion  via  HL7  and  DICOM  MPPS  messages  via 
DICOM  come  across  within  minutes  from  the 
clinical  information  systems. 

Every  hour,  a  report  of  undictated  studies  is 
pulled  from  the  RIS  and  fed  into  a  global  work 
list  for  the  department.  Identified  undictated 
studies  are  then  queried  to  the  PACS  in  real  time 
using  Web  services  to  determine  the  number  of 
images  on  the  PACS  and  the  performing  physi¬ 
cian  for  interventional  studies.  This  method  helps 
alert  staff  to  studies  that  were  completed  but  have 
no  images  on  the  PACS.  Four  times  each  day,  a 
RIS  report  is  pulled  for  a  list  of  unsigned  reports. 
Signing  physicians  are  sent  a  text  page  via  e-mail 
with  the  number  of  unsigned  reports  by  location. 
This  eliminated  an  inefficient  process  in  which 
radiologists  had  to  log  in  to  multiple  RISs  to  see 
whether  a  report  was  ready  to  sign.  Integrating  an 
alerting  mechanism  with  a  dashboard  application 
is  valuable  to  communicate  actionable  informa¬ 
tion  and  ensure  a  timely  feedback  mechanism. 

Information  Visualization 

More  than  a  dozen  commercial  business  intel¬ 
ligence  platforms  that  incorporate  information  vi¬ 
sualization  are  on  the  market  today.  The  business 
intelligence  industry  is  undergoing  rapid  innova¬ 
tion,  with  several  large  vendors  with  complete 


solutions  and  a  multitude  of  smaller  vendors  with 
best-of-breed  components.  The  two  highest-rated 
solutions  as  determined  in  an  industry  analysis  in 
2008  are  BusinessObjects  from  SAP  (Walldorf, 
Germany)  and  Cognos  from  International 
Business  Machines  (Armonk,  NY)  (17). 

We  elected  to  use  a  commercial  Web-based 
graphing  tool  and  construct  our  own  Web  dash¬ 
board  because  of  our  phased  approach  and 
leverage  our  experience  in  Web  development. 

The  dashboard  grew  out  of  several  smaller  qual¬ 
ity  projects  started  in  2004  to  monitor  unsigned 
and  undictated  reports.  The  multiple  projects 
needed  a  roof  to  house  them  all  for  direct  access 
and  comparison.  Although  our  platform  was  de¬ 
veloped  on  open-source  technologies,  significant 
investment  was  required  to  customize  the  dash¬ 
board  to  our  department. 

The  Web  site  was  developed  using  the  open- 
source  programming  language  and  application 
framework  Ruby  on  Rails  (18).  The  Web  site 
used  the  Ruby  ChartDirector  imaging  library 
(Advanced  Software  Engineering,  Hong  Kong) 
to  dynamically  generate  graphics  on  demand 
as  bubble  charts,  histograms,  fuel  gauges,  time 
trends,  and  run  charts.  When  a  user  requests  to 
load  a  Web  page,  the  Ruby  on  Rails  framework 
requests  the  most  recent  data  from  the  data¬ 
base,  which  are  rendered  into  a  graphic  JPEG 
file.  As  the  database  is  updated,  those  changes 
are  immediately  displayed  within  the  graphs. 
Asynchronous  JavaScript  enables  users  to  click 
on  areas  within  the  graph  to  select  subregions. 
This  allows  users  to  intuitively  drill  down 
through  the  graphs  and  explore  complex  dynam¬ 
ics  that  can  affect  operations. 

The  main-level  page  with  the  departmental 
overview  was  distributed  throughout  the  depart¬ 
ment  as  a  Web  link  and  showed  all  the  major  indi¬ 
cators  as  fuel  gauges  (Fig  2). This  provided  novel 
transparencies  within  the  department  to  help 
align  the  focus  on  areas  needing  improvement. 
Thresholds  were  set  for  each  metric  in  the  qual¬ 
ity  meetings  to  define  a  target  zone  in  green,  a 
warning  zone  in  yellow,  and  a  trouble  zone  in  red. 
Many  of  the  performance  targets  were  derived 
from  yearly  departmental  goals  and  objectives. 
The  fuel  gauges  provide  an  easy  method  to  re¬ 
view  all  the  metrics  at  a  glance.  Each  fuel  gauge  is 
linked  to  detailed  analysis  on  that  metric  so  that 
users  can  understand  the  results  more  fully. 
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Figure  2.  The  main  page 
of  the  dashboard  provides 
an  overview  of  the  major  key 
performance  indicators.  Each 
gauge  is  described  in  the  Table. 
C-P  =  completed  study-pre¬ 
liminary  report,  EPR  =  errors 
per  report,  P-F  =  preliminary 
report-final  report,  PICC  =  pe¬ 
ripherally  inserted  central  cath¬ 
eter,  TAT  =  turnaround  time. 
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Figure  3.  Dual-axis  graph 
shows  the  average  outpatient 
waiting  time  for  mammogra¬ 
phy  (green  line)  as  a  function 
of  the  time  of  day  (24-hour 
clock).  Blue  bars  =  patient 
volume. 


A  secure  log-in  is  required  to  access  pages 
that  include  personal  health  information.  We  em¬ 
ployed  the  Lightweight  Directory  Access  Protocol 
for  user  authentication  and  authorization,  based 
out  of  our  PACS.This  allowed  users  to  access  the 
dashboard  with  their  PACS  log-ins  and  eliminated 
the  need  for  separate  password  management. 
According  to  the  sensitivity  of  the  data,  such  as 


radiologist  peer  review,  data  analysis  views  were 
restricted  to  only  those  staff  authorized  for  man¬ 
agement  of  that  metric. 

Beginning  in  July  2006,  the  dashboard  was  used 
as  the  central  focus  of  our  twice-monthly  opera¬ 
tional  meetings.  A  60-inch  (150-cm)  liquid  crystal 
display  was  installed  in  the  main  administrative 
conference  room  so  that  the  dashboard  could  be 
accessed  in  real  time  at  all  administrative  meetings. 
In  addition,  staff  throughout  the  department  were 
given  access  to  the  site  via  a  Web  link. 
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Figure  4.  Dual-axis  graph 
shows  high-priority  inpa¬ 
tient  study  requests  from  the 
time  of  ordering  to  the  time 
of  procedure  completion. 
Blue  bars  =  monthly  volume 
for  one  year,  green  line  = 
average  time  from  order  to 
completion. 


Initial  results  and  subsequent  experience  in¬ 
dicate  that  the  dashboard  improved  our  depart¬ 
ment’s  ability  to  understand  performance  and 
enhanced  participation  in  our  quality  meetings. 

At  the  same  time,  we  were  able  to  improve  our 
departmental  performance  in  several  key  areas. 
Over  a  period  of  1 2  months,  improvements  were 
observed  in  the  radiology  report  turnaround  time, 
outpatient  waiting  times,  stat  order  turnaround 
times,  and  quality  control  resolution  times. 

It  is  challenging  to  attribute  improvements  to 
the  use  of  the  dashboard.  The  dashboard  drove 
discussions  at  the  quality  meetings  and  provided 
context,  but  it  was  the  management  team  who 
used  results  to  initiate  additional  quality  inves¬ 
tigations  into  approaches  for  improving  perfor¬ 
mance.  The  dashboard  did  not  enforce  change; 
instead,  it  enabled  improvement  by  demonstrat¬ 
ing  problem  areas  and  root  causes  for  those  prob¬ 
lems  and  by  helping  align  the  efforts  in  quality 
initiatives.  The  dashboard  also  served  as  a  work¬ 
bench  on  which  new  theories  could  be  tested 
hypothetically  to  determine  whether  they  could 
solve  various  problems. 

Enabling  Knowledge  Discovery 

Each  fuel  gauge  represented  a  key  performance 
indicator.  By  clicking  on  the  fuel  gauge,  the  user 
is  taken  to  an  in-depth  analysis  of  that  indica¬ 
tor.  This  analysis  attempts  to  display  all  the  fac¬ 
tors  that  can  influence  the  indicator  and  may  be 
causes  of  poor  performance.  The  purpose  of  the 
analysis  page  is  to  lead  a  discussion  with  a  suc¬ 
cessive  series  of  questions  that  can  be  answered 
quickly  with  a  graphic  interface.  In  a  traditional 
paper-based  setting,  questions  frequently  cannot 
be  answered  immediately  and  must  be  followed 
up  in  preparation  for  the  next  meeting.  By  pro¬ 


viding  the  ability  to  sift  through  data,  visualize  it 
both  in  detail  and  in  a  graphical  representation, 
and  answer  questions  immediately,  the  dashboard 
not  only  accelerates  the  time  to  fixing  problems 
but  also  leads  a  group  rapidly  through  the  data 
discovery  process,  which  aids  in  their  buy-in  and 
accountability  for  the  analysis  and  results. 

Figure  3  is  an  example  of  one  of  the  levels  of 
analysis  on  our  dashboard  and  shows  average 
outpatient  waiting  times  for  mammography  in 
minutes  (green  line,  plotted  as  a  function  of  the 
hour  of  the  day)  and  number  of  procedures  (blue 
bars) .  This  graph  shows  a  steep  rise  in  waiting 
times  from  1 0  minutes  at  opening  to  more  than 
45  minutes  by  10  am.  This  is  a  common  type  of 
queuing  problem  that  can  be  caused  by  schedul¬ 
ing,  staffing  levels,  room  utilization,  or  examina¬ 
tion  durations.  In  this  example,  the  quality  com¬ 
mittee  determined  that  the  high  waiting  times 
were  the  result  of  scheduling  a  high  number  of 
diagnostic  mammography  procedures  in  the 
morning,  requiring  more  resources  and  longer 
individual  examination  times  than  did  screening 
mammography  procedures. 

Figure  4  shows  the  average  time  from  the  or¬ 
dering  of  a  stat  study  for  an  inpatient  to  the  time 
the  procedure  was  completed.  (Emergency  de¬ 
partment  and  shock  trauma  studies  were  filtered 
out  because  both  areas  have  embedded  imaging 
equipment,  operate  on  an  almost  continuous 
stat  basis,  and  do  not  involve  hospital  patient 
transport.)  The  overview  graph  demonstrates  the 
performance  of  this  indicator  (green  line)  over  a 
period  of  several  months,  overlaid  with  the  num¬ 
ber  of  procedures  performed  (blue  bars).  Other 
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Figure  5.  Bubble  chart  shows 
radiologists’  report  turnaround 
time  for  1  month  by  departmental 
section.  ANG  =  angiography,  CT  = 
computed  tomography,  EXR  = 
emergency  x-ray,  MR1  =  magnetic 
resonance  imaging,  MXR  =  trauma 
x-ray,  NUCMED  =  nuclear  medi¬ 
cine,  PETCT  =  positron  emission 
tomography/CT,  P-F  =  preliminary 
report-final  report,  SONO  =  ultra¬ 
sonography,  USH  =  satellite  hospi¬ 
tal,  XRA  =  x-ray. 


Internal  Perfomance  Analysis 

O  XRA  O  USH  O  SONO  ©  PETCT  ®  NUCMED 


graphs  tied  to  this  indicator  demonstrated  depen¬ 
dent  factors,  such  as  patient  location,  time  of  day, 
day  of  the  week,  requesting  physicians,  requested 
procedure,  and  modality  resources.  In  February 
2008,  a  change  was  made  to  the  order  entry  sys¬ 
tem,  with  ordering  physicians  required  to  enter  a 
pager  number  to  be  called  back  for  high-priority 
orders.  This  change  caused  a  subsequent  drop  in 
orders  with  a  corresponding  improvement  in  re¬ 
sponse  time  for  ordered  studies. 

The  purpose  of  data  collection  and  graphical 
display  is  to  allow  viewers  to  quickly  detect  trends 
or  dips  in  service  and  then  be  able  to  zoom  in  on 
trouble  spots  to  understand  the  cause.  The  pro¬ 
cess  provides  an  overview  of  the  indicator  in  a  dy¬ 
namic  way  that  allows  a  tighter  focus  on  specific 
factors  affecting  performance.  This  allows  the 
user  to  drill  down  through  the  graphs  to  see  the 
underlying  detailed  information  for  each  study, 
thus  helping  explain  specific  problems. 

Figure  5  is  a  report  turnaround  bubble  chart 
displaying  the  sections  within  the  department.  A 
bubble  chart  is  a  useful  information  visualization 
tool  for  displaying  three  dimensions  of  data.  The 
x  axis  represents  the  average  time  from  the  com¬ 
pletion  of  the  study  to  the  time  of  the  preliminary 
report.  The  y  axis  is  the  time  from  preliminary 
signing  to  final  signing.  Each  section  is  repre¬ 
sented  by  a  different  color,  and  the  size  of  each 
bubble  is  directly  related  to  the  number  of  proce¬ 
dures  that  section  performed  for  that  month. 


Figure  6  is  a  trending  analysis  showing  depart¬ 
mental  radiologist  reporting  time  over  2  years  as 
a  percentage  of  studies  in  the  department  that 
were  reported  within  2-,  8-,  24-,  and  48-hour 
windows.  The  timeliness  of  radiology  reporting 
has  received  significant  attention  in  the  process 
improvement  literature  (19,20).  Our  department 
has  conducted  several  initiatives  to  reduce  the 
turnaround  time,  and  the  dashboard  was  able 
to  plot  the  effects.  The  department  began  auto¬ 
matically  paging  radiologists  daily  for  unsigned 
reports  in  March  2005,  implemented  speech  rec¬ 
ognition  throughout  2006,  and  in  January  2007 
instituted  a  new  system  to  synchronize  study  sta¬ 
tus  between  the  RIS  and  the  PACS.  A  graphical 
dashboard  can  act  as  an  instrument  on  which  to 
conduct  experiments  in  process  improvement. 

Conclusions 

Radiology  management  is  under  considerable 
pressure  to  do  more  with  less  (21).  A  perfect 
storm  is  forming,  created  by  declining  reimburse¬ 
ment  rates,  rising  expectations  of  patients  and  cli¬ 
nicians  for  faster  turnaround  times,  and  national 
initiatives  to  improve  the  quality  in  radiology  to 
enhance  outcomes.  These  forces  are  driving  radi¬ 
ology  professionals  to  look  for  ways  to  improve 
efficiency  and  productivity  without  sacrificing 
quality.  Business  analytics  is  a  proven  tool  to  help 
a  business  become  more  competitive  (22). 

Using  informatics  extraction  techniques,  we 
were  able  to  capture  the  majority  of  indicators 
routinely  used  in  our  quality  meetings  and  drive 
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Figure  6.  Graph  shows 
the  2-year  progression  in  the 
percentage  of  reports  that 
were  signed  promptly  in  the 
radiology  department  after 
implementation  of  automated 
quality  control  and  dash¬ 
boarding  tools. 


the  process  in  a  paperless,  automated  fashion. 
This  was  viewed  as  a  significant  improvement 
that  made  the  meetings  more  effective  and  pro¬ 
vided  a  better  understanding  of  the  operations 
of  the  department. 

Significant  effort  was  required  in  extraction 
of  the  metrics  from  clinical  information  systems. 
Although  some  vendors  provide  reporting  pack¬ 
ages  and  a  few  provide  a  Web  service  program¬ 
ming  interface,  no  standard  methods  have  been 
defined  for  extracting  metrics  in  radiology.  We 
have  added  components  to  our  purchase  require¬ 
ments  for  new  clinical  information  systems  that 
request  the  vendor  to  expose  internal  operational 
business  logic  using  a  service-oriented  architec¬ 
ture  based  on  Web  services.  This  allows  other 
systems  to  gather  data  for  business  intelligence 
applications.  We  encourage  other  consumers  of 
clinical  information  systems  to  consider  their  in¬ 
teroperability  needs  for  business  intelligence. 

For  an  energized  leadership  committed  to  ser¬ 
vice  excellence  and  quality,  a  dashboard  can  be 
a  powerful  tool  to  help  improve  performance  in 
radiology.  Continuous  quality  efforts  are  widely 
recognized  as  crucial  elements  in  the  successful 


radiology  department  (23).  However,  quality  com¬ 
mittees  and  leadership  too  often  lack  the  tools  to 
effectively  drive  change  and  remain  relevant. 

A  Web-based  graphical  dashboard  provides  a 
level  of  transparency  of  operations  to  empower 
effective  management.  Management  in  our  de¬ 
partment  found  this  to  be  a  useful  alignment 
tool  by  exploring  data  as  a  group  on  a  projected 
screen.  Supervisors,  section  chiefs,  and  adminis¬ 
trators  were  provided  with  access  to  the  Web  site. 
Another  important  benefit  has  been  the  reduc¬ 
tion  in  the  effort  and  time  previously  required  to 
collect  and  prepare  reports. 

Acknowledgment:  The  authors  thank  Nancy  Knight, 
PhD,  for  her  expert  assistance  in  preparing  the  manu¬ 
script. 
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T  he  two  principal  benefits  of  a  business  intelligence  solution  that  can  affect  the  culture  of  an 
organization  are  transparency  and  fact-based  decision  making  (2). 
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The  time  and  effort  required  to  build  reports  creates  a  capture  latency  that  diminishes  the  value  of  the 
information  as  well  as  the  time  the  organization  then  has  to  take  action  to  identify  and  implement 
remedies  (2). 
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Paper-based  reporting  can  answer  only  a  relatively  small  list  of  questions  that  must  be  identified 
before  the  monthly  (or  other  periodic)  meeting  that  routinely  addresses  such  questions. 
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Paper-based  reporting  lacks  the  granularity  to  drill  down  on  a  metric  and  view  the  original  data.  T  he 
ability  to  view  the  actual  data  from  which  statistical  analyses  are  derived  offers  a  higher  level  of 
confidence  in  understanding  data  and  making  subsequent  decisions. 
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Continuous  quality  improvement  requires  the  ability  to  change  the  metrics  on  the  dashboard  as  well 
as  the  targets  for  those  metrics. 
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Appendix  C.  Innovations  in  the  Surgical  Environment  Retreat  Agenda 
8:30  Welcome  and  Introductions,  charge  to  the  group  Dr.  Park 
Brief  review  of  ISE  history  Dr.  Park 
9:15  Key  findings,  pre-retreat  interviews  Dr.  Gordon 
Discussion  of  results  Dr.  Park 
10:00  Break 
1 1:00  Discussion 
Ranking  exercise 
12:30  Working  lunch 
1:30  Work  groups  (Break  out) 

1.  Govemance/Organization 

2.  Funding 

3.  Program 

4.  Research 

2:30  Group  reports  and  discussion 
3:30  Summary  and  next  steps 
4:00  Adjourn 

Distinguished  Participants  in  the  ISE  Planning  Retreat 
Caroline  Cao,  MD,  PhD  Tufts  University 
LTC  Jerome  Buller,  MD  Bethesda  Naval 
Ken  Dobler  Ethicon  Endo  Surgery 
Erik  Dutson,  MD  USC  Stanford 
Robert  Foster,  PhD  US  DoD  ret. 

Dennis  Fowler,  MD,  MPH  Columbia  University 
Ivan  George  University  of  Maryland 
Toby  Gordon,  ScD  Gordon  and  Associates  /  JHU 
Greg  Hager,  PhD  Johns  Hopkins  University 
Rosemary  Klein,  MA  University  of  Maryland 
Gyusung  Lee,  PhD  University  of  Maryland 
Reuben  Mezrich,  MD,  PhD  University  of  Maryland 
Gerry  Moses  University  of  Maryland 
Paul  Nagy,  PhD  University  of  Maryland 
Adrian  Park,  MD  University  of  Maryland 

(COL)  Ron  Poropatich,  MD  US  Army  TATRC  (Diverted  from  attending) 

Steve  Schwaitzburg,  MD  Cambridge  Health  Alliance 
Richard  Satava,  MD  University  of  Washington  /  DoD  ret. 

Raj  Shekhar,  PhD  Children’s  Washington  DC 
Brady  Shirley  IMD  Medical,  Inc. 

General  Fred  Smalkin,  JD,  BG  US  Army  ret. 

Russ  Taylor,  PhD  Johns  Hopkins  University 

Yelena  Yesha,  PhD  UMBC  (Univ.  MD  Baltimore  Campus) 
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D.  Summation  of  ISE  Expert  Workgroups 


Summation  of  ISE  Expert  Workgroup  Activity 

The  strategic  planning  group  then  divided  into  four  discussion  panels.  Each  considered  a 
topic  -  Program,  Funding,  Research,  and  Organization/Governance  -in  regard  to  ISE 
expansion  and  was  charged  with  bringing  back  3  levels — operational;  strategic  and 
BHAG — of  recommendations. 

Program  Group  (Schwaitzberg  -  leader;  George,  Buller,  Caban,  Dutson): 

Strategic:  The  group  suggested  that  future  meetings  might  incorporate  a  “resort  model” 
with  an  aim  to  ensure  pairing  of  interventionalist  and  non-clinician.  The  thought  was  that 
the  program  should  be  looking  to  mate  with  the  organization,  i.e.,  as  ISE  looks  to  “set  the 
dialogue”  the  program  be  developed  on  the  dialogue  of  the  moment.  A  mix  of  presenters 
fluent  and  expert  in  the  “current  dialogue”  should  be  recruited.  A  specific  focus  on 
establishing  individual  relationships  with  well-established  and  exciting  innovators  such 
as  Steve  Jobs,  Bill  and  Melinda  Gates,  etc.  that  would  result  in  drawing  high  caliber  (and 
a  large  volume  of)  attendees  to  ISE  events  was  identified  as  crucial.  Meeting  frequency 
was  considered  to  be  tied  to  funding,  but  an  annual,  if  possible,  event  was  considered 
optimal. 

Operational:  The  program  committee  would,  in  the  main,  be  responsible  for  obtaining 
presenters  as  well  as  scouting  out  and  establishing  high-level  innovative  relationships.  In 
order  to  specifically  establish  how  the  next  large  fonnat  ISE  program  might  appear,  it 
was  suggested  that  ISE  hold  a  consensus  conference  over  two  days,  with  one  day  devoted 
to  clarifying  “Vision”  and  the  other  to  clarifying  “Academics.”  One  discussion  that 
ensued  in  the  group  meeting  was  in  regard  to  ISE  not  being  about  clinical  disease  but 
rather  about  the  environment  in  which  clinical  disease  is  treated;  this  would  be  among  the 
types  of  Issues  that  would  be  dealt  with  at  the  consensus  conference.  The  group 
detennined  that  based  on  the  current  regulatory  and  fiscal  environment  ISE  would  not 
offer  CME.  A  side  discussion  resulted  in  the  suggestion  that  the  organization’s  title  be 
changed  to  the  Institute  of  Interventional  and  Surgical  Medicine  (IoISM). 

BHAG:  The  group  favored  a  large  conference  with  “big  name”  speakers  injected  into  the 
program  and  a  program  that  is  firmly  grounded  in  setting  and  carrying  on  the  dialogue. 
Significant  discussion  (per  Dr.  Schwaitzberg)  centered  on  holding  an  ISE  equivalent  of  a 
TED  conference.  Essentially,  the  conference  as  envisioned  by  the  group  is  one  that 
generates  a  large  “buzz”  and  large  interest  and  a  large  participant  pool,  though  with  a 
degree  of  exclusivity. 

Funding  Group  (Yesha  -  leader;  Moses,  Taylor  and  Smalkin): 

Strategic:  The  overall  group  would  recommend,  as  expected  given  previously  held 
strategic  discussion,  the  fonnation  of  a  fonnal  organization  to  support  and  extend  the  ISE 
conference  and  to  serve  as  a  clearing  house  for  policy  development  and  funding.  The 
need  was  stated  for  a  small  cadre  of  permanent  staff  led  by  an  Executive  Director  to 
sustain  the  effort.  Major  costs  would  be  incurred  in  regard  to  the  scheduling  and 
conducting  of  the  annual  ISE  conference.  The  group  approached  the  issue  of  fiscal 
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support  with  a  broad  brush,  noting  that  many  options  should  be  developed  to  obtain 
funding. 

Operational:  Operational  costs  and  smaller  grants  will  be  sought  with  NIST,  TATRC, 
and  AHRQ  identified  as  specific  agency  targets.  Broader  funding  needs  would  be 
approached  via  direct  appeals  for  donor  support,  proposals  to  government  and  provincial 
granting  agencies,  foundation  grants,  and  possible  industry  sponsorship  (The  IBM  Center 
for  Advanced  Studies  and  also  insurance  companies  were  specifically  proposed  for 
funding  targeting).  Establishment  of  a  small  cadre  of  permanent  staff  led  by  an  Executive 
Director  would  serve  several  functions  1]  support  of  the  formal  organization,  2]  support 
and  expansion  of  the  ISE  conference,  and  3]  coordination  of  collaborative  activities.  As 
broadly  and  widely  as  possible,  international  (beyond  North  America)  people  and  ideas 
should  be  included.  We  should  take  advantage  of  the  current  national  focus  on  Health  IT 
and  try  to  leverage  smart  health  prevention  and  prediction  with  the  surgical  innovation. 
BHAG:  The  best  model  for  long-tenn  sustainability  of  ISE  may  be  to  create  a  non-profit 
organization  and  later  create  for-profit  company  (possibly  based  at  the  University)  that 
will  operate  the  non-profit  aspect  and  can  also  deal  with  Ventures.  Yesha  should 
approach  IBM  and  seek  sponsorship  for  the  ISE  conference  and  try  to  co-locate  ISE  with 
GASCON  2011.  This  year  is  the  100th  Anniversary  of  IBM,  so  we  may  get  some  more 
funding  and  publicity.  The  event  will  be  in  Toronto  Nov.2-5,  2011.  Another  option  would 
be  to  host  the  conference  in  Halifax  and  ask  NSERC  for  support. 

Research  Group  (Cao  -  leader;  Klein,  Hager,  Lee,  Satava  and  Shekhar): 

Strategic:  The  ISE  will  systematically  generate  innovation.  In  the  near  term,  the  ISE 
membership  (given  its  combined  expertise  and  brainpower)  will  work  to  identify  clinical 
needs.  These  needs  will  then  be  addressed  at  focused  meetings  to  generate  new  ideas.  We 
will  invite  top  researchers  in  the  identified  areas  to  these  meetings  to  present  their  ideas. 
In  so  doing,  ISE  can  strategically  “harvest  technology,”  vet  ideas,  and  support  the 
feasibility  studies  and  eventual  development  of  these  new  and  emerging  technologies. 
Domain  knowledge  possession  then  would  be  a  role  (ISE  knows  how  to  translate 
technology  into  use”)  Initial,  seed  funding  may  be  provided  to  the  investigators  to 
prototype  their  ideas.  Resources  or  simulation  facilities  may  also  be  provided  to  ensure 
maximal  output  in  a  short  time  frame  to  demonstrate  the  potential  of  these  new  ideas. 
Intellectual  Property  (IP)  becomes  a  significant  issue  in  any  discussion  of  research; 
among  the  issues  are  “invention  disclosure”  and  “public  domain.”  Any  intellectual 
property  will  belong  to  the  investigators.  ISE  will  not  own  any  IP.  ISE  should,  in  Gao’s 
terms,  consider  “Partnering  for  Intellectual  Energy.” 

Operational:  ISE  can  leverage  its  mandate  of  creating  physician-driven/  physician- 
centered  needs-driven  innovation  into  partnership  with  other  professional  groups, 
especially  the  engineering  profession  (e.g.,  IEEE,  HFES,  IEA,  etc.).  This  will  allow  ISE 
to  become  the  focal  point  of  a  network  of  professional  associations,  facilitating  cross¬ 
society  pollination  and  collaboration,  and  ultimately  being  the  beneficiary  of  the  research 
efforts.  Going  forward  ISE  needs  to  take  into  account  the  question  “Who  is  the  group 
you’re  pitching  to?”  and  it  needs  a  person  or  team  to  drive  this  project,  much  as  MMVR 
is  TATRC-  and  DARPA-driven.  ISE  also  needs  one  success  story  showing  how  the  ISE 
model  in  regard  to  a  demonstrable  pilot  study  works.  The  ISE  Conference  of  the  future 
was  described  as  a  venue  “to  harvest  ideas  w/  a  package  to  be  bought.”  The  Conference 
would  also  serve  to  bring  together  thought  leaders  and  innovators  from  different 
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disciplines  to  brainstonn,  come  up  with  a  concept,  develop  a  pathway,  realize  a 
prototype.  Work  would  be  published,  a  web  site  maintained,  and  new  intellectual 
opportunities  -  Systems-Oriented  Concept  or  End-to-End  Holistic  -  created.  It  was  also 
suggested  that  ISE  could  offer  workshops  of  the  type  that  were  focused  such  as  the 
Hopkins  Summer  Workshop  Series  where  one  thing  is  worked  on  and  where  an  idea  can 
be  scrutinized  by  a  panel  of  experts  and  that  could  be  tied  into  an  organization  such  as 
MMVR  (it  was  suggested  that  ISE  could  offer  surgical  training  workshops,  supportable 
by  TATRC  conference  monies  ($15  to  20,000  for  brand-new  interesting  ideas). 

BHAG:  The  question  is  can  ISE  shepherd  a  project  from  conception  to  clinical 
implementation  by  removing  all  obstacles  such  as  funding,  clinician  acceptance, 
regulatory  requirements,  etc.  (Fairly  early  on  the  point  was  made  that  the  ISE  group  — 
while  “not  capable  of  the  ‘full  research  process’”  —  could  harvest  technology  components 
with  an  emphasis  on  emerging  technology  and  basic  simple  components.)  Another 
BHAG  noted  by  Satava  was  that  ISE  would  eventually  face  operationalization  in  a 
number  (3  or  4)  of  hospitals. 

Research  Pillars 

The  4  or  5  pillars  for  ISE  research  cannot  be  replaced  but  need  to  be  re-defined  as  they 
can  be  misleading.  Most  of  the  research  issues  for  ISE  are  cross-cutting,  i.e.,  they  result 
from  a  combination  of  several  pillars,  not  least  of  which  is  a  human  factors  pillar.  In  the 
absence  of  a  better  nomenclature,  use  of  the  tenninology  “pillars”  serves  as  a 
categorizing  device  for  workshops’  and/or  conferences’  themes. 


Research  Approach 

The  OR  is  a  complex  socio-technical  environment,  utilizing  a  synthesis  of  technologies 

that  include  everything  from  small  mechanical  devices  to  complex  infonnation  systems, 

from  techniques  to  processes.  Therefore,  we  recommend  a  systems  approach  which  takes 

into  account  any  themes  represented  as  research  pillars  with  a  focus  on  innovation  in  the 

OR  (such  as  ISE  developing  a  new  paradigm  in  surgical  approach). 

Organization/Governance  Group  (Fowler  -  leader;  Dobler,  Mezrich,  Park  and 
Shirley): 

Strategic:  Structure:  The  group  agreed  unanimously  that  we  need  a  formal  organization 
to  define  the  mission  and  execute  the  goals  of  ISE.  The  concept  is  that  the  ISE  would  be  a 
consortium  (or  at  least  consortium-like)  and  would  include  constituents  from  academia, 
industry,  the  military,  other  government  entities,  and  other  organizations.  The  latter 
could  include  organizations  as  diverse  as  IEEE  and  AORN.  The  fonnal  structure  would 
likely  include  officers,  a  board,  committees,  and  a  hired  executive  director  to 
operationalize  both  the  structure  and  function  of  the  ISE.  Ideally,  the  board  would  consist 
of  someone  sympathetic  to  the  needs/goals  of  at  least  one  of  the  multiple,  diverse 
constituent  groups. 
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Audience:  The  group  spent  more  time  discussing  the  audience  than  any  other  topic. 
Ultimately,  we  agreed  that  the  audience  might  include  providers  (both  individual  and 
institutional),  medic al/surgical  societies,  technology  startups,  big  industry,  NIH,  AHRQ, 
payers,  and  policymakers. 

Mission  and  Goals:  Based  on  the  thought  that  the  design  process  is  flawed,  ISE  should 
strive  to  facilitate  innovation  through  support  of  new  ideas  and  also  facilitate  their 
development  through  the  applied  research  and  commercialization  phases  of  technology 
development. 

Operational:  Tactical:  The  committee  structure  of  ISE  should  be  developed  in  response 
to  the  goals  of  the  organization.  Working  committees  would  design  and  deliver  on  each 
goal  of  the  organization.  This  was  actually  started  in  the  retreat  with  the  Program  and 
Funding  committees. 

BHAG:  ISE  should  approach  the  Institute  of  Medicine  and  seek  to  become  their  arm 
engaged  in  overseeing  innovation  and  improvement  in  the  interventional  cycle.  By 
combining  the  expertise  of  the  ISE  membership  with  the  political  clout  and  gravitas  of 
the  IOM,  the  impact  on  the  intervention  cycle  in  both  the  near-  and  long-term  future 
could  be  very  significant. 


42 


